surfra
Run your model on serverless GPUs and CPUs across many clouds. You own the runtime. Surfra runs the queue, the replicas, and the failover.
Salad
RunPod
Vast.ai
Yotta
DigitalOcean
Your machines
Surfra queues requests, starts replicas on the machines you rank, moves to the next provider when one is full, and stops replicas when they go idle.
Load your model once in a warmup. Return JSON from a handler. Or skip the handler and run vLLM, SGLang, TGI, or Ollama as they are.
import surfra model = None @surfra.warmup def load(): global model model = load_my_model() @surfra.handler def predict(request): return {"text": model.generate( request.input["prompt"])}
List GPUs in priority order. A new replica takes the first row that can place it, so a full provider falls through to the next.
SaladRTX 4090FullRunPodRTX 4090PlacingYour machineRTX 3090ReadyEvery call goes through the queue. It starts replicas when none are running and is retried if a worker fails.
import surfra app = surfra.App("my-model") print(app.run({"prompt": "Say hi"})) # {'text': 'Hi!'} # or any OpenAI client client = OpenAI( base_url="https://api.surfra.io/openai/my-llm/v1", api_key=SURFRA_API_KEY, )
Queued work starts replicas. When the first provider is full, the next one takes the overflow. Idle replicas stop after the idle timeout, so a quiet app scales to zero.
Queued work starts replicas. Idle replicas stop after the idle timeout. Set a minimum to keep some warm.
A full or missing provider falls through to the next row. A request that fails on one worker is retried on another.
Per-provider limits keep a base on one provider and send bursts to another.
Write a handler, or run a server you already have. An HTTP app also answers OpenAI-compatible calls.
Chain apps into steps that pass their output forward. Each step's app scales on its own, so a GPU model and a CPU post-processor share one workflow.
Register a workstation or a spare server. It shows up as a row in any app's priority list, next to the clouds.
Each app has metrics, a request log with inputs and outputs, live logs, worker states, and a release history.
Invite people to an organization. Apps, credentials, and the balance are shared. Everyone keeps their own API keys.
Add funds to a prepaid balance. A replica is charged for the seconds it runs, and you see the price before you deploy.
Surfra runs your model on GPUs and CPUs from many clouds as one serverless app. You bring a container image. Surfra queues requests, starts replicas on the machines you rank, moves to the next provider when one is full, and stops replicas when they go idle.
No. Add a small handler with the surfra Python package, or run the server you already use, such as vLLM, SGLang, TGI, or Ollama, behind the Surfra supervisor.
Salad, RunPod, Vast.ai, Yotta, and DigitalOcean today, plus machines you register yourself. You choose the order, and Surfra falls through the list when a provider has no room.
You keep a prepaid balance and pay for the seconds a replica runs. Top ups start at $25, and you can refill automatically with a saved card. Machines stop when the balance reaches $0.
Yes. An organization shares apps, credentials, and one balance. Owners and admins invite people, manage billing, and remove members. Each person has their own API keys.
Surfra is invite only while we test it. Email hello@surfra.io and tell us what you want to run.