How to Combine GPU Capacity Across Providers

TL;DR

To combine GPU capacity across providers, run a separate cluster per provider and schedule jobs across them, use snapshot migration for one-time moves, or run each GPU pool as a location of one virtual cloud so an inference service keeps serving when a pool goes dark. Control Plane takes the third approach. One workload runs active-active across locations on different clouds, a global endpoint sends each request to the nearest healthy location, and workloads running at least two replicas in at least two locations are covered by a 99.999% SLA. GPU Ocean lets you pull GPU capacity from any provider into the same virtual cloud.

GPU capacity rarely sits in one place. You may hold reserved H100s at one provider, spot A100s at another, a few T4s or A10Gs on a hyperscaler, and a rack of your own servers. Each pool has its own console, identity model, and network. The question is how to treat them as one pool without rebuilding your stack every time a contract or price changes. The three approaches below solve different problems, and the right one depends on whether you run batch training, always-on inference, or both.

The three approaches at a glance

Approach 1: Run a separate cluster per provider

The default is a Kubernetes or Slurm cluster at each provider, with jobs submitted to whichever cluster has free GPUs, by hand or through a scheduler.

Where it works well. Checkpointed training runs, sweeps, and batch jobs that can wait in a queue or restart from a checkpoint on another cluster.

Where it stops. Each cluster keeps its own console, IAM, credentials, private network, and observability. Nothing routes users across providers, so an inference service lives in one cluster and goes down with it. Every new provider adds another stack to secure and operate.

Approach 2: Snapshot migration

Snapshot migration moves a workload when capacity moves. You capture the container image, model weights, and any state, then restore them on the new provider.

Where it works well. Planned moves: leaving a provider at contract end, consolidating pools, or relocating to a region with better pricing. It is also the simplest model to reason about for stateful jobs.

Where it stops.

• Model weights are large. A 70B model in FP16 is about 140 GB, so each cross-provider copy costs roughly $12 to $17 in list-price internet egress on AWS, GCP, or Azure, and takes minutes to hours depending on throughput. Copying to several pools, or moving training checkpoints, multiplies both.
• Every target needs its own networking, secrets, and identity setup, so the snapshot captures only part of what the workload depends on.
• Drift accumulates. The restored copy and the original diverge unless you automate the rebuild.
• As a failover plan it is reactive. You migrate after capacity disappears, and the service is down or degraded until the restore finishes.

Treat snapshots as a way to move data, not as the operating model for a multi-provider fleet.

Approach 3: Run GPU pools as locations of one virtual cloud

The third approach changes the question. Instead of moving workloads between providers, you make each GPU pool a location the same workload can run in, and let the platform keep it serving.

In Control Plane, a location is a managed cloud region, a Kubernetes cluster you already run, or a cluster of your own servers. A Global Virtual Cloud (GVC) groups locations, and a workload deployed to the GVC runs in every one of them at once. Every workload gets a global endpoint that sends each request to the nearest healthy location, so when a pool fails, traffic moves to the others without a runbook (docs). The 99.999% availability SLA covers workloads running at least two replicas in at least two locations. During the October 20, 2025 AWS outage in us-east-1, affected Control Plane workloads were active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane internal incident telemetry (Beyond Backups).

• Capacity is a placement choice. Adding a location to the GVC provisions the workload there, and removing one drains its traffic to the rest. Neither touches the workload definition.
• One identity model. Universal Cloud Identity gives workloads short-lived access to 600+ services across AWS, GCP, and Azure, such as the bucket holding your model weights, with no long-lived cloud keys in your code.
• One network. Services in the GVC reach each other over mutual TLS across locations, and the firewall denies traffic by default until you allow it.
• One place to watch. Logs and metrics from every location land in one observability stack.

GPU supply. On Control Plane’s managed regions, the workload spec takes NVIDIA T4 (1 to 4 per replica) and A10G (1 per replica), with drivers installed and kept current by Control Plane (docs). The pay-as-you-go GPU catalog goes further, from T4 and L4 through L40S, A100, H100, H200, and B200, priced per GPU-hour with per-second billing, subject to availability (pricing). GPU Ocean lets you pull GPU capacity from any provider, and you can do it today with Managed Kubernetes by joining GPU servers from any provider as locations. Clusters you register bring their own accelerators, and non-NVIDIA cards are requested by their Kubernetes resource name.

Inspectiv runs its custom AI inference layer on Control Plane-sourced GPUs. As its CTO, Ross Hendrickson, put it: “I don’t have to vet another vendor or go through compliance and security just to get access to specific GPUs.” (case study)

Which approach should you choose for GPU workloads?

• Inference that has to stay up across providers, or a fleet you expect to keep reshaping: run the pools as locations of a virtual cloud. It is the only one of the three approaches that adds a shared identity model, a private network with mutual TLS, and nearest-healthy routing across providers.
• Batch jobs that can wait for capacity: separate clusters with a job queue work. Scheduled single-node GPU jobs, such as batch inference, embeddings, and evaluation, also run on Control Plane as cron workloads in the same virtual cloud as your inference.
• A one-time provider exit or consolidation: use snapshot migration, and move weights through object storage close to the target.
• Both training and serving: the approaches compose. Train where capacity is cheapest, then serve the model from a virtual cloud so the serving layer is not tied to wherever training happened to land.

Blueprint: GPU inference across locations on Control Plane

This blueprint serves one model from NVIDIA T4 GPUs in two managed locations on different clouds, AWS and GCP, active-active, then shows how to add a GPU cluster you run yourself. GPU access and the GPU-enabled locations for your org are set up by Control Plane support.

Step 1: Register each GPU pool as a location

• Managed cloud regions. Use built-in locations such as aws-us-east-2. A new org starts with a working set of them, and support adds any region you are missing.
• A Kubernetes cluster you already run, such as a neocloud GPU cluster. Label at least one node group cpln.io/nodeType=core. In the console, create a location, choose Actions, Install, and run the generated kubectl command against the cluster within about five and a half minutes, because the manifest carries short-lived tokens. Confirm the agent is running with kubectl get pod -l app=cpln-byok-agent -n kube-system (docs).
• Your own GPU servers. Create a Managed Kubernetes cluster with the Generic provider and run the generated join script on each server. Then create a location and enable the cluster’s CPLN Platform add-on pointing at it, which installs the agent for you (docs).

Step 2: Create a GVC that spans the locations

kind: gvc
name: inference
spec:
  staticPlacement:
    locationLinks:
      - //location/aws-us-east-2
      - //location/gcp-us-east1

Step 3: Deploy the inference workload once

kind: workload
name: llm-inference
gvc: inference
spec:
  type: serverless
  identityLink: //identity/inference-identity   # access to the bucket with your weights
  containers:
    - name: server
      image: your-registry/your-inference-server:latest
      cpu: "4"        # a T4 with more than 14Gi of memory needs 4 cores per GPU
      memory: 16Gi    # a T4 allows 7Gi to 31Gi per GPU
      ports:
        - number: 8000
          protocol: http
      gpu:
        nvidia:
          model: t4
          quantity: 1
  defaultOptions:
    capacityAI: false
    autoscaling:
      metric: concurrency
      target: 8
      minScale: 2
      maxScale: 5     # the default quota is 5 replicas per workload; ask support for more
  firewallConfig:
    external:
      inboundAllowCIDR:
        - 0.0.0.0/0
      outboundAllowHostname:
        - "*.amazonaws.com"   # wherever the model weights live

Create the identity first, with a cloud access rule for the bucket that holds your weights (identity docs), then apply both files with cpln apply -f gvc.yaml and cpln apply -f workload.yaml.

• type: serverless, because only serverless workloads autoscale on concurrent requests, which suits long, uneven inference calls.
• capacityAI: false, because Capacity AI right-sizing does not apply to a container with a GPU. The card sets the minimum CPU and memory, so size the rest deliberately.
• minScale: 2 keeps two warm replicas in each location, as the docs recommend for production. A serverless workload can scale to zero with minScale: 0 where cold starts are acceptable, but the SLA covers workloads running at least two replicas in at least two locations.
• firewallConfig, because Control Plane denies inbound and outbound traffic by default. Open inbound for the public endpoint and outbound only to the hosts that hold your weights.

Step 4: Keep weights close to each pool

Large weights pulled across the internet on every cold start cost time and egress. Control Plane can mount an S3, GCS, or Azure Blob bucket into the container as a read-only file system, so the inference server loads weights without a custom download step. Keep that bucket close to the location that serves the most traffic, and for distant locations bake the weights into the image or cache them on local disk at startup. This is the part of snapshot migration worth keeping.

Step 5: Add a GPU cluster you run yourself

A cluster you register advertises its own accelerators, and a container requests them by resource name in gpu.custom, for example amd.com/gpu, while managed regions use gpu.nvidia. A container sets one or the other, so run your own cluster as its own GVC with a copy of the workload that changes only the GPU block. The image, identity model, firewall policy, and observability stay the same, and Control Plane operates both deployments through the same API, CLI, and console.

Step 6: Test failover before you need it

• Suspend the workload in one location with location-specific options and confirm the global endpoint keeps serving from the others.
• Load test each location to confirm it scales out.
• Measure cold-start time per location, including weight load, and set minScale to match your latency target.
• Watch logs and metrics for every location from one place.

• Track cost per million tokens by location, to decide where warm replicas pay for themselves.

Frequently asked questions

How do you combine GPU capacity from multiple cloud providers?
Use one of three approaches. Separate clusters per provider can run batch jobs wherever GPUs are free. Snapshot migration moves a workload to a new provider when capacity moves. Running each GPU pool as a location of one virtual cloud keeps an inference service serving across providers. Control Plane takes the third approach.

What is the best way to run GPU inference across providers?
Run the GPU pools as locations of one virtual cloud and deploy the inference workload once. On Control Plane, the workload runs active-active in every location of its Global Virtual Cloud, a global endpoint sends each request to the nearest healthy location, and the 99.999% SLA covers workloads running at least two replicas in at least two locations.

Which GPUs does Control Plane offer?
On managed regions, the workload spec takes NVIDIA T4 and A10G. The pay-as-you-go catalog lists NVIDIA cards from T4 and L4 through L40S, A100, H100, H200, and B200, priced per GPU-hour with per-second billing, subject to availability. GPU Ocean lets you pull GPU capacity from any provider, today through Managed Kubernetes, and clusters you register bring their own accelerators.

Can I use my own GPU servers with Control Plane?
Yes. Join the servers to a Managed Kubernetes cluster with the Generic provider, or register a Kubernetes cluster you already run, and each becomes a location in a Control Plane Global Virtual Cloud. Workloads there request the accelerators the cluster advertises.

Next step: Sign up free and deploy a GPU workload to two locations, or talk to the Control Plane team about pooling the GPU capacity you already have.