Container Orchestration Beyond Kubernetes

Why looking past the default is key to right‑sizing deployments and reducing complexity in 2025

When I first learned Kubernetes, it felt like a superpower. Pods, deployments, services, and Ingress gave me a consistent way to ship apps at scale. Over the years, it became the default answer for almost everything. But the longer I work with containers in production, the more I realize “Kubernetes by default” can be the wrong answer. Sometimes it’s overkill. Sometimes it’s a cost sink. Sometimes the operational overhead devours the developer velocity you were chasing in the first place.

In this post, I’ll walk through the landscape beyond Kubernetes: orchestration models that are simpler, cheaper, and sometimes more focused. We’ll look at where they shine, where they struggle, and where the tradeoffs actually matter. I’ll ground the discussion in real patterns and code you can try, and I’ll share personal lessons from projects where picking the right orchestration layer saved us from both under‑engineering and over‑engineering. If you’re a developer or a curious engineer wondering whether Kubernetes is the only path to production, this is for you.

Where orchestration fits today: real teams and real projects

Kubernetes is dominant for good reasons: portability, a rich ecosystem, and a powerful control plane. In large organizations, platform teams use it to standardize deployments across dozens of services. In startups, it often arrives as “the safe choice” when scaling beyond a single host. For teams with complex microservices, multi‑tenant requirements, or strict SLAs, it’s frequently the right call.

But it’s not the only call. In the real world, I see teams reaching for simpler options when:

• They’re running a small number of services with predictable traffic.
• They’re hosting on a single cloud provider and don’t need portability.
• They want to minimize operational toil and keep platform costs low.
• They need a fast feedback loop and aren’t yet ready for a dedicated platform team.

Alternatives like Docker Compose for local development and simple staging, Nomad for flexible workload scheduling, and managed services like AWS App Runner or Fly.io for PaaS‑style deployments are increasingly common. Cloud runtimes such as Google Cloud Run and Azure Container Apps provide serverless containers that scale to zero, which can be a huge cost win for low‑traffic services.

Compared to Kubernetes, these options usually trade breadth for simplicity. They often have gentler learning curves, fewer moving parts, and lower infrastructure overhead. They’re less flexible at the edges, but that’s frequently a feature, not a bug. The key is to map your constraints to the right model rather than defaulting to the most popular tool.

Technical core: practical models and patterns beyond Kubernetes

Docker Compose for simple multi‑container apps

Docker Compose isn’t a cluster orchestrator, but it’s a foundational piece of the puzzle. It’s ideal for local development, testing, and small deployments where you don’t need distributed scheduling or autoscaling. The workflow is straightforward: define services, networks, and volumes in YAML, then bring them up with a single command.

Here’s a realistic Compose setup for a web API, a Postgres database, and a Redis cache. It includes health checks and a shared network, making it a good base for local development or a simple staging environment.

version: "3.9"

services:
  api:
    build:
      context: ./app
      dockerfile: Dockerfile
    ports:
      - "8080:8080"
    environment:
      - DATABASE_URL=postgres://app:password@db:5432/app
      - REDIS_URL=redis://cache:6379
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 10s
      timeout: 5s
      retries: 5

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_USER=app
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=app
    volumes:
      - pgdata:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 10s
      timeout: 5s
      retries: 5

  cache:
    image: redis:7-alpine
    ports:
      - "6379:6379"

volumes:
  pgdata:

In practice, using Compose like this lets you iterate quickly on the API while keeping the data layer consistent. When you need to go beyond a single host, you can migrate to a cluster orchestrator or a managed service. A common pattern is to keep Compose for local dev and a production‑ready compose file for ephemeral staging environments, while using a separate deployment model for production.

Nomad: flexible scheduling for mixed workloads

Nomad is an open source scheduler from HashiCorp that supports containers, VMs, and even binary workloads. It’s simpler than Kubernetes, but it’s not toy software. I’ve used it to run containerized services alongside batch jobs and periodic tasks, all within the same cluster. Its job spec is approachable and the operational footprint is small.

Below is a Nomad job spec for a web service with multiple allocations (tasks), health checks, and a service discovery block. Nomad integrates with Consul for service registration, which is useful for internal routing.

job "webapp" {
  datacenters = ["dc1"]
  type        = "service"

  group "web" {
    count = 3

    network {
      port "http" {
        to = 8080
      }
    }

    service {
      name = "webapp"
      port = "http"

      tags = ["web", "app"]

      check {
        type     = "http"
        path     = "/health"
        interval = "10s"
        timeout  = "5s"
      }
    }

    task "api" {
      driver = "docker"

      config {
        image = "myregistry/webapp:1.4.0"
        ports = ["http"]
        auth {
          username = "${NOMAD_META_DOCKER_USER}"
          password = "${NOMAD_META_DOCKER_PASS}"
        }
      }

      env {
        DATABASE_URL = "postgres://app:password@db.internal:5432/app"
        REDIS_URL    = "redis://cache.internal:6379"
      }

      resources {
        cpu    = 500
        memory = 256
      }

      restart {
        attempts = 3
        interval = "2m"
        delay    = "15s"
        mode     = "delay"
      }
    }
  }
}

What makes Nomad appealing in production is the operational simplicity. You get multi‑region support, upgrades that don’t require re‑architecting your applications, and the ability to run non‑container workloads if needed. For teams that don’t need the Kubernetes ecosystem’s breadth, it’s a pragmatic scheduler with a friendly learning curve.

Cloud‑native PaaS: AWS App Runner, Google Cloud Run, Azure Container Apps

Managed PaaS offerings abstract away cluster management entirely. You point to a container image, configure environment variables, and let the platform handle scaling and availability. These are fantastic for services that don’t require complex scheduling or bespoke networking.

The following Dockerfile is a simple Node.js API. It’s deliberately minimal so that it’s easy to deploy anywhere, including PaaS offerings. The health endpoint is built in, and the app listens on the expected port.

# syntax=docker/dockerfile:1
FROM node:20-alpine

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

COPY . .

EXPOSE 8080
CMD ["node", "server.js"]

A small Node server with a health endpoint and an environment‑driven port:

// server.js
const http = require('http');
const url = require('url');

const PORT = process.env.PORT || 8080;

const server = http.createServer((req, res) => {
  const { pathname } = url.parse(req.url);

  if (pathname === '/health') {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ status: 'ok' }));
    return;
  }

  if (pathname === '/') {
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end('Hello from containerized API\n');
    return;
  }

  res.writeHead(404);
  res.end('Not Found');
});

server.listen(PORT, () => {
  console.log(`listening on :${PORT}`);
});

Deploying this to AWS App Runner or Google Cloud Run is typically done via the CLI or console. For example, App Runner can build from source or use an ECR image. Cloud Run expects a container image and lets you configure concurrency and CPU allocation. The developer experience is smooth: fast deploys, built‑in HTTPS, and autoscaling with scale‑to‑zero on many platforms.

The tradeoff is flexibility. You won’t run DaemonSets or custom operators. You may have constraints on networking, background workers, or specialized hardware. But for many web services and APIs, these platforms are ideal.

Fly.io: edge‑deployed containers with a simple model

Fly.io is a platform that runs containers close to users, with a straightforward CLI and configuration. It’s particularly nice for small teams that want a global footprint without managing multiple regions. The fly.toml file controls routing, health checks, and scaling, which keeps the deployment configuration close to the code.

Here’s a sample fly.toml for a web service with basic health checks and auto‑restart policies:

app = "my-webapp"

[build]
  dockerfile = "Dockerfile"

[[services]]
  internal_port = 8080
  protocol = "tcp"

  [[services.ports]]
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.http_checks]]
    path = "/health"
    interval = 10000
    grace_period = "5s"
    timeout = "2s"

[[restart]]
  policy = "on-failure"
  max_retries = 3

The mental model on Fly is closer to “one process per container, scaled horizontally.” It’s a good fit for web apps and lightweight workers. It may not be the best choice for complex stateful services or large, heterogeneous clusters, but it’s a compelling option for fast iteration and global reach.

Raspberry Pi cluster: learning and edge scenarios

For a hands‑on, low‑cost way to explore orchestration, a small Raspberry Pi cluster is invaluable. I’ve used a four‑node Pi cluster to simulate edge deployments and test failover patterns. It’s a great place to experiment with Nomad or Docker Swarm, especially when you want to understand scheduling and service discovery without a cloud bill.

A minimal Docker Swarm stack looks like this. It defines a web service and a database, with placement constraints and health checks. Swarm is built into Docker, so the learning curve is gentle if you already use Compose.

version: "3.9"

services:
  web:
    image: myregistry/webapp:1.4.0
    ports:
      - "80:8080"
    deploy:
      replicas: 3
      restart_policy:
        condition: on-failure
        delay: 15s
        max_attempts: 3
      update_config:
        parallelism: 1
        delay: 10s
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 10s
      timeout: 5s
      retries: 3

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_USER=app
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=app
    volumes:
      - db-data:/var/lib/postgresql/data
    deploy:
      placement:
        constraints:
          - node.role == manager

volumes:
  db-data:

Swarm’s simplicity makes it a good “first step” into clustering. It won’t replace Kubernetes for complex needs, but for small clusters or edge deployments, it gets the job done.

Honest evaluation: strengths, weaknesses, and tradeoffs

Kubernetes:

• Strengths: Extremely flexible, massive ecosystem, strong community, portable across clouds, rich scheduling and policy features, operators for complex stateful workloads.
• Weaknesses: Steep learning curve, high operational overhead, easy to over‑engineer, costs can spiral without careful management.
• Good for: Microservices platforms, teams with dedicated platform engineers, multi‑cloud requirements, advanced scheduling and policy needs.

Nomad:

• Strengths: Simple job spec, low operational overhead, supports non‑container workloads, multi‑region friendly, easy to run on small clusters.
• Weaknesses: Smaller ecosystem, fewer managed offerings, less community momentum than Kubernetes.
• Good for: Mixed workloads, teams that want a lightweight scheduler, environments where operational simplicity is a priority.

Docker Compose:

• Strengths: Easy to write and understand, great for local development, quick prototyping.
• Weaknesses: Not a distributed scheduler, no native autoscaling, limited production readiness without additional tooling.
• Good for: Local dev, simple staging environments, small deployments.

Cloud PaaS (App Runner, Cloud Run, Container Apps):

• Strengths: Fast deployments, managed scaling, low operational burden, built‑in HTTPS and observability.
• Weaknesses: Less flexibility, platform‑specific constraints, potential vendor lock‑in, limited for specialized workloads.
• Good for: APIs, web apps, low‑traffic services, teams without a dedicated platform team.

Fly.io:

• Strengths: Global edge deployments, simple configuration, developer‑friendly CLI.
• Weaknesses: Not ideal for complex stateful services, smaller ecosystem than major cloud platforms.
• Good for: Global web apps, small teams, rapid iteration.

Docker Swarm:

• Strengths: Integrated with Docker, simple to set up, good for small clusters.
• Weaknesses: Limited feature set compared to Kubernetes, smaller community.
• Good for: Learning, small clusters, edge deployments.

When to choose what:

• If you have a small number of services and need a fast, low‑cost path to production, start with a PaaS or Fly.io.
• If you’re local‑first or building a simple staging environment, use Compose.
• If you need a flexible scheduler for mixed workloads without Kubernetes complexity, consider Nomad.
• If you need the full power of operators, advanced networking, and multi‑cloud portability, Kubernetes is likely the right choice.

Personal experience: lessons from real projects

I once helped a team migrate from a sprawling Kubernetes setup to a simpler architecture for a low‑traffic internal tool. We had overprovisioned a cluster, added Prometheus, Grafana, Loki, Istio, and a handful of operators, only to find that our “platform” took more time to maintain than the app took to build. We moved the service to Google Cloud Run, kept the same container image, and trimmed our monthly infrastructure cost by more than 70 percent. The deployment pipeline became two commands, and the team refocused on features.

On the other hand, I’ve also seen the opposite. Another project started on Docker Compose and quickly hit limits: uneven traffic, a need for better reliability, and a desire to run background workers alongside the API. We moved to Nomad because Kubernetes felt like a sledgehammer for the job. Nomad’s job spec was easy for developers to read, and the ops overhead was minimal. We kept our cognitive load low while still gaining scheduling and service discovery.

A common mistake I see is picking a tool for a future state that may never arrive. Starting with Kubernetes “because we might scale” often leads to complexity before you need it. A better approach is to pick the simplest tool that meets your current constraints, and evolve when the pain is real. Another mistake is ignoring the learning curve. Even if a tool is “simple,” the ecosystem and operational patterns take time to master. Plan for that in your roadmap.

The moment I realized the value of looking beyond Kubernetes was when a production incident turned into a five‑minute fix instead of a multi‑hour debugging session. The service was on Cloud Run, and a bad deploy was automatically rolled back due to health check failures. No pods stuck in CrashLoopBackOff, no node pressure, no CNI mysteries. Just a clean rollback and a chance to fix the bug. It wasn’t a victory of complexity; it was a victory of simplicity.

Getting started: workflow, tooling, and mental models

Your workflow should match your constraints. For local development, Compose is the foundation. For deployment, choose the simplest model that meets reliability and scalability requirements. Avoid mixing orchestration models unless you have a clear reason.

A simple project structure for a Compose‑first workflow:

my-app/
├── app/
│   ├── Dockerfile
│   ├── server.js
│   └── package.json
├── docker-compose.yml
├── docker-compose.prod.yml   # optional, for a production-like staging
├── .github/
│   └── workflows/
│       └── deploy.yml        # GitHub Actions or similar
└── README.md

A production‑like staging compose file might override the database to use a managed instance, or switch to a production image registry. Keep parity between local and staging as high as possible without incurring unnecessary costs.

For Nomad, a simple local setup is a single node agent in dev mode. You can run jobs, inspect allocations, and learn the CLI without a complex cluster. In production, run a small cluster with Consul for service discovery. Keep jobs simple, with clear resource limits and health checks. Avoid over‑committing CPU and memory; Nomad is efficient, but realistic reservations prevent noisy neighbor issues.

For PaaS, the mental model is “image in, URL out.” Focus on configuring environment variables, secrets, and health checks. Use the platform’s observability tools and avoid reinventing logging or metrics pipelines until you have a clear need. Keep deploys small and frequent, and rely on built‑in rollbacks.

In all cases, prioritize:

• Health checks that reflect real user experience.
• Environment parity for configuration and secrets.
• Simple deployment pipelines that are easy to debug.
• Observability that matches your team’s skills and needs.

What makes these alternatives stand out

Compared to Kubernetes, the alternatives often excel in developer experience and maintainability. They reduce the surface area for misconfiguration and keep the deployment story close to the code. They’re easier to teach to new team members, and they usually have fewer moving parts to break.

The ecosystem around Kubernetes is powerful, but that power comes with complexity. Operators, CRDs, and networking plugins are fantastic when you need them, but they can distract from delivering features if you don’t. In contrast, tools like Nomad, Compose, and PaaS platforms focus on the basics: run your containers, expose them to users, and keep them healthy. That simplicity translates into faster iteration and fewer surprises in production.

Free learning resources

• Nomad Introduction and Job Specification: https://developer.hashicorp.com/nomad/docs
• Docker Compose File Reference: https://docs.docker.com/compose/compose-file/
• AWS App Runner Documentation: https://docs.aws.amazon.com/apprunner/
• Google Cloud Run Documentation: https://cloud.google.com/run/docs
• Azure Container Apps Documentation: https://learn.microsoft.com/azure/container-apps/
• Fly.io Documentation: https://fly.io/docs/
• Raspberry Pi Cluster Guides: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html

These resources provide hands‑on examples and reference material. They’re useful because they focus on practical deployment patterns, configuration details, and real‑world constraints.

Summary: who should use what, and when to skip

Pick Kubernetes if you need advanced scheduling, multi‑cloud portability, or a broad ecosystem of operators and tools. It’s the right choice for complex microservice platforms and organizations with dedicated platform teams. If you don’t have those needs, it’s worth considering simpler options.

Use Nomad when you want a flexible scheduler with a low operational burden. It’s great for mixed workloads and teams that prefer a straightforward job spec over a complex API.

Start with Docker Compose for local development and small deployments. It’s a safe, low‑friction way to build and test multi‑container applications.

Choose cloud PaaS or Fly.io when you want to ship quickly and minimize ops. These are excellent for web APIs, low‑traffic services, and teams that value developer velocity over platform flexibility.

If you’re learning, a Raspberry Pi cluster or a small Nomad/Docker Swarm setup provides a realistic environment without a big cloud bill.

The takeaway is simple: match the orchestration model to your current constraints, not an imagined future. Kubernetes is powerful, but it’s not the only path to production. Sometimes the best engineering decision is to pick the tool that gets you to a reliable, maintainable system with the least complexity. That’s how you keep your team focused on shipping, not babysitting infrastructure.