Chuyển đến nội dung
MZGrid

Trang này chỉ có bằng tiếng Anh. Phiên bản tiếng Anh là bản có giá trị chính thức.

Templates

Every workload template — what it runs, minimum VRAM, recommended GPUs and an example request.

Templates are pre-built container configurations. Pass the template id as template when creating a job. The live list, including image references, exposed ports and recommended SKUs, is available from GET /v1/catalog/templates.

ID mẫuTênLoạiVRAM tối thiểu
llama-3.1-8bLlama 3.1 8B Instructendpoint suy luận24 GB
llama-3.3-70bLlama 3.3 70B Instruct FP8endpoint suy luận80 GB
qwen2.5-32bQwen2.5 32B Instruct AWQendpoint suy luận24 GB
vllm-openaivLLM OpenAI-compatibleendpoint suy luận16 GB
sdxl-comfyuiComfyUI + SDXL / FLUX.1-schnelltạo ảnh16 GB
whisper-large-v3Whisper large-v3endpoint suy luận10 GB
pytorch-jupyterPyTorch 2.5 + CUDA 12.4 JupyterLabtương tác8 GB
custom-dockerCustom Docker imagetùy chỉnhbất kỳ
batchBatch jobhàng loạtbất kỳ

The scheduler only considers offers whose VRAM is at least the template's minimum. Recommended SKUs are the GPUs we expect to give a good balance of price and performance; any GPU at or above the minimum will run.

vllm-openai — vLLM OpenAI-compatible endpoint

Serves any Hugging Face model supported by vLLM behind the OpenAI Chat Completions and Completions API. Set the model with config.model and pass extra vLLM flags in config.args.

  • Min VRAM: 16 GB
  • Recommended: rtx4090, l40s, a100-80, h100-sxm depending on model size
JSON
{
  "sku": "l40s",
  "autoSelect": true,
  "template": "vllm-openai",
  "maxDurationHours": 8,
  "config": {
    "model": "mistralai/Mistral-7B-Instruct-v0.3",
    "args": ["--max-model-len", "32768"],
    "env": { "HF_TOKEN": "hf_..." }
  }
}

llama-3.1-8b — Llama 3.1 8B Instruct (vLLM)

Llama 3.1 8B Instruct in BF16 with vLLM's OpenAI-compatible server. A fast, inexpensive general-purpose chat model. Requires a Hugging Face token with access to the gated model.

  • Min VRAM: 24 GB
  • Recommended: rtx4090, rtx3090, rtx5090, l40s
JSON
{
  "sku": "rtx4090",
  "autoSelect": true,
  "template": "llama-3.1-8b",
  "maxDurationHours": 2,
  "config": { "env": { "HF_TOKEN": "hf_..." } }
}

llama-3.3-70b — Llama 3.3 70B Instruct FP8 (vLLM)

Llama 3.3 70B Instruct quantised to FP8, which fits on a single 80 GB GPU while leaving room for the KV cache. Hopper and Blackwell GPUs run FP8 natively.

  • Min VRAM: 80 GB
  • Recommended: h100-sxm, h100-pcie, h200, b200
JSON
{
  "sku": "h100-sxm",
  "autoSelect": true,
  "template": "llama-3.3-70b",
  "maxDurationHours": 4,
  "config": {
    "args": ["--max-model-len", "16384"],
    "env": { "HF_TOKEN": "hf_..." }
  }
}

qwen2.5-32b — Qwen2.5 32B Instruct AWQ (vLLM)

Qwen2.5 32B Instruct with 4-bit AWQ weights. Strong reasoning and coding quality that fits on a single 24 GB card.

  • Min VRAM: 24 GB
  • Recommended: rtx4090, rtx5090, l40s, rtx6000-ada, a6000
JSON
{
  "sku": "rtx5090",
  "autoSelect": true,
  "template": "qwen2.5-32b",
  "maxDurationHours": 6
}

On 24 GB GPUs keep the context modest, for example "args": ["--max-model-len", "8192"]. 48 GB cards allow much longer contexts.

sdxl-comfyui — ComfyUI + SDXL / FLUX.1-schnell

ComfyUI with SDXL and FLUX.1-schnell checkpoints preloaded. The endpoint URL opens the ComfyUI web interface, and the ComfyUI HTTP API is available on the same port for automated pipelines.

  • Min VRAM: 16 GB
  • Recommended: rtx4090, rtx5090, rtx4080s, l40s
JSON
{
  "sku": "rtx4090",
  "autoSelect": true,
  "template": "sdxl-comfyui",
  "maxDurationHours": 3
}

whisper-large-v3 — Whisper transcription API

OpenAI Whisper large-v3 behind an HTTP transcription API compatible with the OpenAI /v1/audio/transcriptions request format.

  • Min VRAM: 10 GB
  • Recommended: rtx4080s, rtx3090, rtx4090
JSON
{
  "sku": "rtx4080s",
  "autoSelect": true,
  "template": "whisper-large-v3",
  "maxDurationHours": 1
}
Shell
curl "$ENDPOINT_URL/v1/audio/transcriptions" \
  -F [email protected] \
  -F model=whisper-large-v3

pytorch-jupyter — PyTorch 2.5 + CUDA 12.4 JupyterLab

An interactive JupyterLab environment with PyTorch 2.5 and CUDA 12.4. The endpoint URL opens JupyterLab; the access token is printed to the job log at start-up. Files in the working directory live on the job's scratch storage and are deleted when the job ends — copy anything you want to keep to your own storage before stopping.

  • Min VRAM: 8 GB
  • Recommended: any; rtx4090 or a100-80 for model development
JSON
{
  "sku": "a100-80",
  "autoSelect": true,
  "template": "pytorch-jupyter",
  "maxDurationHours": 5
}

custom-docker — Custom Docker image

Runs any public container image that uses the NVIDIA runtime. Specify config.image and the ports to publish. Use this for your own inference servers, fine-tuning services or tools that are not covered by the other templates.

  • Min VRAM: any (pick a SKU that fits your workload)
JSON
{
  "offerId": "off_3h8v0rz2pa",
  "template": "custom-docker",
  "maxDurationHours": 12,
  "config": {
    "image": "ghcr.io/acme/embeddings-server:2.1.0",
    "ports": [8080],
    "env": { "MODEL_ID": "BAAI/bge-large-en-v1.5" }
  }
}

Images must be pullable without registry credentials. Read the Docker documentation on building GPU images and the NVIDIA Container Toolkit docs for runtime requirements.

batch — Batch job

Runs a container to completion and settles when it exits. You are charged for the seconds it ran; the rest of the deposit is refunded. Use it for evaluation runs, embedding a dataset, rendering or short fine-tuning jobs. The exit code is reported with the result.

  • Min VRAM: any
JSON
{
  "sku": "h100-pcie",
  "autoSelect": true,
  "template": "batch",
  "maxDurationHours": 3,
  "config": {
    "image": "ghcr.io/acme/eval-harness:0.9",
    "args": ["--tasks", "gsm8k,mmlu", "--output", "s3://acme-evals/run-112"],
    "env": { "AWS_ACCESS_KEY_ID": "...", "AWS_SECRET_ACCESS_KEY": "..." }
  }
}

Cảnh báo

Environment variables are delivered only to the job's container, but that container runs on a third-party provider's machine. Use short-lived, narrowly scoped credentials. Read the Security model before sending secrets.