We moved 200+ microservices from AWS to multi-region GCP over 6 months. Not a single request dropped. Here's what that actually looked like, including the parts that nearly broke us.

MPL (Mobile Premier League) was running at 5 million concurrent users during peak tournaments. The infrastructure was on AWS, and it worked. But "it works" is a low bar when you're adding 200-400ms of unnecessary latency because users in India and Brazil are hitting US-East endpoints. A GCP partnership brought the opportunity to rethink the whole setup.

The fear was simple: one bad config, one race condition in the database migration, one missed dependency — and millions of live game sessions crash simultaneously. That's not a pager alert. That's a press release.

The migration was 80% preparation and 20% execution. Everything that went wrong, went wrong during preparation — when we could afford it. By the time we touched production traffic, we had run the playbook so many times there was nothing left to surprise us.

The Strategy: Traffic Mirroring, Not Big Bang

We ruled out the big-bang approach immediately. "Pick a weekend and flip DNS" is how you get an incident on Monday morning and a retrospective nobody wants to write.

Instead: deploy everything to GCP in parallel, mirror traffic, then shift gradually using weighted DNS. The AWS clusters kept running. The GCP clusters got real traffic at increasing percentages — 5%, then 20%, then 50%, then 100%. At any point we could roll back by editing two numbers in a Terraform file. 

# Week 1: 5% GCP, 95% AWS
resource "google_dns_record_set" "api" {
  routing_policy {
    wrr {
      weight  = 95
      rrdatas = [aws_alb_endpoint]
    }
    wrr {
      weight  = 5
      rrdatas = [gcp_lb_endpoint]
    }
  }
}
# Week 3: flip to 50/50
# Week 6: 100% GCP, AWS decommissioned

Every service had a cloud_provider label in Prometheus. Grafana dashboards showed error rate, latency P99, and saturation split by AWS vs GCP in real time. Any degradation on GCP was visible at 5% traffic exposure, not after the full cutover.

The Terraform Journey: 32 Modules and State Management Nightmares

Before touching any traffic, we spent six weeks Terraforming the entire GCP environment. Every GKE cluster, VPC, subnet, IAM binding, Cloud DNS zone, Cloud Armor policy, NAT gateway: all code. We built 32+ reusable modules with regional targeting so the same module could deploy to India, US, and Brazil with different variable files. 

module "gke_cluster_india" {
  source       = "./modules/gke-cluster"
  project_id   = var.project_id
  region       = "asia-south1"
  cluster_name = "prod-india"
  node_pools   = var.india_node_pools
  network      = google_compute_network.platform_vpc.name
  subnetwork   = google_compute_subnetwork.india.name
}

module "gke_cluster_brazil" {
  source       = "./modules/gke-cluster"
  region       = "southamerica-east1"
  cluster_name = "prod-brazil"
  node_pools   = var.brazil_node_pools
  # same module, different vars
}

The state management was the ugly part. We started with a single state file per environment. Sounds fine until you have 32 modules and a plan that takes 8 minutes to run and touches things it has no business touching. We eventually split into per-module state with remote backends on GCS. Design your state boundaries before you have 600 resources in one file.

Terraform workspaces are great in theory. In practice, if you add them after you already have production state, you're in for a painful migration. Start with them from day one.

ArgoCD App-of-Apps: Managing 200 Services Across 3 Clusters

Deploying 200+ microservices to three GKE clusters by hand was never an option. We used ArgoCD's app-of-apps pattern with ApplicationSets: one parent ArgoCD app per cluster region, generating child apps from a directory structure in our GitOps repo. 

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: platform-services
spec:
  generators:
    - git:
        repoURL: https://github.com/org/gitops-repo
        revision: HEAD
        directories:
          - path: apps/platform/*
  template:
    metadata:
      name: '{{path.basename}}'
    spec:
      project: platform
      source:
        repoURL: https://github.com/org/gitops-repo
        targetRevision: HEAD
        path: '{{path}}'
      destination:
        server: '{{cluster}}'
        namespace: platform
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

New service? Add a directory, push to main, ArgoCD deploys it to all three clusters within minutes. No manual kubectl, no per-cluster ceremony. This was the unlock that made 200+ services manageable.

KEDA and the Python Session Controller

MPL's traffic is violently spiky. 8pm India time is tournament prime time, and concurrent users can double in under a minute. AWS's managed autoscaling handled this adequately. GCP's default HPA did not: it scaled on CPU, which lagged real demand by 3-5 minutes.

We set up KEDA with a custom Python session controller that exposed active session counts as custom metrics. KEDA's ScaledObject pointed at these metrics and kept one pod per 250 active sessions. Scale-up happened in under 30 seconds. Scale-down waited for sessions to drain before evicting. The autoscaler finally understood what it was actually scaling.

The Night We Flipped DNS

Six months in. All 200 services deployed on GCP. Stateful migrations complete: PostgreSQL replicas promoted, Kafka Mirror Maker 2 offset sync verified, Redis AOF dumps restored. The weighted DNS had been at 50/50 for two weeks without a single incident.

We scheduled the final flip for a Tuesday at 2am IST, lowest traffic in the week. Change DNS weights to 100% GCP, verify for 30 minutes, decommission AWS load balancers. Four people on a call, dashboards open, fingers on the rollback Terraform.

The weight change propagated in 47 seconds (we'd set TTL to 60). Latency dropped. Error rate didn't move. The GCP clusters absorbed 100% of production traffic like they'd been doing it for years. In a sense, they had. Thirty minutes later, we decommissioned the last AWS load balancer. Total production impact: zero.

The Numbers

200+
Services migrated
20%
Cost reduction
30%
Latency drop (India/Brazil)
0
Incidents during migration

What We'd Do Differently

The migration wasn't glamorous. Six months of dual environments, incremental traffic shifts, and dashboards left open 24/7. When the final DNS change went out and nothing broke, it felt exactly right. Not dramatic. Just done.