Esta página está disponível apenas em inglês. A versão em inglês é a que prevalece.
Templates
Every workload template — what it runs, minimum VRAM, recommended GPUs and an example request.
Templates are pre-built container configurations. Pass the template id as template when creating a job. The live list, including image references, exposed ports and recommended SKUs, is available from GET /v1/catalog/templates.
| ID do template | Nome | Tipo | VRAM mín. |
|---|---|---|---|
| llama-3.1-8b | Llama 3.1 8B Instruct | endpoint de inferência | 24 GB |
| llama-3.3-70b | Llama 3.3 70B Instruct FP8 | endpoint de inferência | 80 GB |
| qwen2.5-32b | Qwen2.5 32B Instruct AWQ | endpoint de inferência | 24 GB |
| vllm-openai | vLLM OpenAI-compatible | endpoint de inferência | 16 GB |
| sdxl-comfyui | ComfyUI + SDXL / FLUX.1-schnell | geração de imagens | 16 GB |
| whisper-large-v3 | Whisper large-v3 | endpoint de inferência | 10 GB |
| pytorch-jupyter | PyTorch 2.5 + CUDA 12.4 JupyterLab | interativo | 8 GB |
| custom-docker | Custom Docker image | personalizado | qualquer |
| batch | Batch job | lote | qualquer |
The scheduler only considers offers whose VRAM is at least the template's minimum. Recommended SKUs are the GPUs we expect to give a good balance of price and performance; any GPU at or above the minimum will run.
vllm-openai — vLLM OpenAI-compatible endpoint
Serves any Hugging Face model supported by vLLM behind the OpenAI Chat Completions and Completions API. Set the model with config.model and pass extra vLLM flags in config.args.
- Min VRAM: 16 GB
- Recommended:
rtx4090,l40s,a100-80,h100-sxmdepending on model size
{
"sku": "l40s",
"autoSelect": true,
"template": "vllm-openai",
"maxDurationHours": 8,
"config": {
"model": "mistralai/Mistral-7B-Instruct-v0.3",
"args": ["--max-model-len", "32768"],
"env": { "HF_TOKEN": "hf_..." }
}
}llama-3.1-8b — Llama 3.1 8B Instruct (vLLM)
Llama 3.1 8B Instruct in BF16 with vLLM's OpenAI-compatible server. A fast, inexpensive general-purpose chat model. Requires a Hugging Face token with access to the gated model.
- Min VRAM: 24 GB
- Recommended:
rtx4090,rtx3090,rtx5090,l40s
{
"sku": "rtx4090",
"autoSelect": true,
"template": "llama-3.1-8b",
"maxDurationHours": 2,
"config": { "env": { "HF_TOKEN": "hf_..." } }
}llama-3.3-70b — Llama 3.3 70B Instruct FP8 (vLLM)
Llama 3.3 70B Instruct quantised to FP8, which fits on a single 80 GB GPU while leaving room for the KV cache. Hopper and Blackwell GPUs run FP8 natively.
- Min VRAM: 80 GB
- Recommended:
h100-sxm,h100-pcie,h200,b200
{
"sku": "h100-sxm",
"autoSelect": true,
"template": "llama-3.3-70b",
"maxDurationHours": 4,
"config": {
"args": ["--max-model-len", "16384"],
"env": { "HF_TOKEN": "hf_..." }
}
}qwen2.5-32b — Qwen2.5 32B Instruct AWQ (vLLM)
Qwen2.5 32B Instruct with 4-bit AWQ weights. Strong reasoning and coding quality that fits on a single 24 GB card.
- Min VRAM: 24 GB
- Recommended:
rtx4090,rtx5090,l40s,rtx6000-ada,a6000
{
"sku": "rtx5090",
"autoSelect": true,
"template": "qwen2.5-32b",
"maxDurationHours": 6
}On 24 GB GPUs keep the context modest, for example "args": ["--max-model-len", "8192"]. 48 GB cards allow much longer contexts.
sdxl-comfyui — ComfyUI + SDXL / FLUX.1-schnell
ComfyUI with SDXL and FLUX.1-schnell checkpoints preloaded. The endpoint URL opens the ComfyUI web interface, and the ComfyUI HTTP API is available on the same port for automated pipelines.
- Min VRAM: 16 GB
- Recommended:
rtx4090,rtx5090,rtx4080s,l40s
{
"sku": "rtx4090",
"autoSelect": true,
"template": "sdxl-comfyui",
"maxDurationHours": 3
}whisper-large-v3 — Whisper transcription API
OpenAI Whisper large-v3 behind an HTTP transcription API compatible with the OpenAI /v1/audio/transcriptions request format.
- Min VRAM: 10 GB
- Recommended:
rtx4080s,rtx3090,rtx4090
{
"sku": "rtx4080s",
"autoSelect": true,
"template": "whisper-large-v3",
"maxDurationHours": 1
}curl "$ENDPOINT_URL/v1/audio/transcriptions" \
-F [email protected] \
-F model=whisper-large-v3pytorch-jupyter — PyTorch 2.5 + CUDA 12.4 JupyterLab
An interactive JupyterLab environment with PyTorch 2.5 and CUDA 12.4. The endpoint URL opens JupyterLab; the access token is printed to the job log at start-up. Files in the working directory live on the job's scratch storage and are deleted when the job ends — copy anything you want to keep to your own storage before stopping.
- Min VRAM: 8 GB
- Recommended: any;
rtx4090ora100-80for model development
{
"sku": "a100-80",
"autoSelect": true,
"template": "pytorch-jupyter",
"maxDurationHours": 5
}custom-docker — Custom Docker image
Runs any public container image that uses the NVIDIA runtime. Specify config.image and the ports to publish. Use this for your own inference servers, fine-tuning services or tools that are not covered by the other templates.
- Min VRAM: any (pick a SKU that fits your workload)
{
"offerId": "off_3h8v0rz2pa",
"template": "custom-docker",
"maxDurationHours": 12,
"config": {
"image": "ghcr.io/acme/embeddings-server:2.1.0",
"ports": [8080],
"env": { "MODEL_ID": "BAAI/bge-large-en-v1.5" }
}
}Images must be pullable without registry credentials. Read the Docker documentation on building GPU images and the NVIDIA Container Toolkit docs for runtime requirements.
batch — Batch job
Runs a container to completion and settles when it exits. You are charged for the seconds it ran; the rest of the deposit is refunded. Use it for evaluation runs, embedding a dataset, rendering or short fine-tuning jobs. The exit code is reported with the result.
- Min VRAM: any
{
"sku": "h100-pcie",
"autoSelect": true,
"template": "batch",
"maxDurationHours": 3,
"config": {
"image": "ghcr.io/acme/eval-harness:0.9",
"args": ["--tasks", "gsm8k,mmlu", "--output", "s3://acme-evals/run-112"],
"env": { "AWS_ACCESS_KEY_ID": "...", "AWS_SECRET_ACCESS_KEY": "..." }
}
}Aviso
Environment variables are delivered only to the job's container, but that container runs on a third-party provider's machine. Use short-lived, narrowly scoped credentials. Read the Security model before sending secrets.