본문으로 건너뛰기
MZGrid

이 페이지는 영어로만 제공됩니다. 영어 버전이 정본으로서 효력을 가집니다.

Templates

Every workload template — what it runs, minimum VRAM, recommended GPUs and an example request.

Templates are pre-built container configurations. Pass the template id as template when creating a job. The live list, including image references, exposed ports and recommended SKUs, is available from GET /v1/catalog/templates.

템플릿 ID이름유형최소 VRAM
llama-3.1-8bLlama 3.1 8B Instruct추론 엔드포인트24 GB
llama-3.3-70bLlama 3.3 70B Instruct FP8추론 엔드포인트80 GB
qwen2.5-32bQwen2.5 32B Instruct AWQ추론 엔드포인트24 GB
vllm-openaivLLM OpenAI-compatible추론 엔드포인트16 GB
sdxl-comfyuiComfyUI + SDXL / FLUX.1-schnell이미지 생성16 GB
whisper-large-v3Whisper large-v3추론 엔드포인트10 GB
pytorch-jupyterPyTorch 2.5 + CUDA 12.4 JupyterLab대화형8 GB
custom-dockerCustom Docker image사용자 지정제한 없음
batchBatch job배치제한 없음

The scheduler only considers offers whose VRAM is at least the template's minimum. Recommended SKUs are the GPUs we expect to give a good balance of price and performance; any GPU at or above the minimum will run.

vllm-openai — vLLM OpenAI-compatible endpoint

Serves any Hugging Face model supported by vLLM behind the OpenAI Chat Completions and Completions API. Set the model with config.model and pass extra vLLM flags in config.args.

  • Min VRAM: 16 GB
  • Recommended: rtx4090, l40s, a100-80, h100-sxm depending on model size
JSON
{
  "sku": "l40s",
  "autoSelect": true,
  "template": "vllm-openai",
  "maxDurationHours": 8,
  "config": {
    "model": "mistralai/Mistral-7B-Instruct-v0.3",
    "args": ["--max-model-len", "32768"],
    "env": { "HF_TOKEN": "hf_..." }
  }
}

llama-3.1-8b — Llama 3.1 8B Instruct (vLLM)

Llama 3.1 8B Instruct in BF16 with vLLM's OpenAI-compatible server. A fast, inexpensive general-purpose chat model. Requires a Hugging Face token with access to the gated model.

  • Min VRAM: 24 GB
  • Recommended: rtx4090, rtx3090, rtx5090, l40s
JSON
{
  "sku": "rtx4090",
  "autoSelect": true,
  "template": "llama-3.1-8b",
  "maxDurationHours": 2,
  "config": { "env": { "HF_TOKEN": "hf_..." } }
}

llama-3.3-70b — Llama 3.3 70B Instruct FP8 (vLLM)

Llama 3.3 70B Instruct quantised to FP8, which fits on a single 80 GB GPU while leaving room for the KV cache. Hopper and Blackwell GPUs run FP8 natively.

  • Min VRAM: 80 GB
  • Recommended: h100-sxm, h100-pcie, h200, b200
JSON
{
  "sku": "h100-sxm",
  "autoSelect": true,
  "template": "llama-3.3-70b",
  "maxDurationHours": 4,
  "config": {
    "args": ["--max-model-len", "16384"],
    "env": { "HF_TOKEN": "hf_..." }
  }
}

qwen2.5-32b — Qwen2.5 32B Instruct AWQ (vLLM)

Qwen2.5 32B Instruct with 4-bit AWQ weights. Strong reasoning and coding quality that fits on a single 24 GB card.

  • Min VRAM: 24 GB
  • Recommended: rtx4090, rtx5090, l40s, rtx6000-ada, a6000
JSON
{
  "sku": "rtx5090",
  "autoSelect": true,
  "template": "qwen2.5-32b",
  "maxDurationHours": 6
}

On 24 GB GPUs keep the context modest, for example "args": ["--max-model-len", "8192"]. 48 GB cards allow much longer contexts.

sdxl-comfyui — ComfyUI + SDXL / FLUX.1-schnell

ComfyUI with SDXL and FLUX.1-schnell checkpoints preloaded. The endpoint URL opens the ComfyUI web interface, and the ComfyUI HTTP API is available on the same port for automated pipelines.

  • Min VRAM: 16 GB
  • Recommended: rtx4090, rtx5090, rtx4080s, l40s
JSON
{
  "sku": "rtx4090",
  "autoSelect": true,
  "template": "sdxl-comfyui",
  "maxDurationHours": 3
}

whisper-large-v3 — Whisper transcription API

OpenAI Whisper large-v3 behind an HTTP transcription API compatible with the OpenAI /v1/audio/transcriptions request format.

  • Min VRAM: 10 GB
  • Recommended: rtx4080s, rtx3090, rtx4090
JSON
{
  "sku": "rtx4080s",
  "autoSelect": true,
  "template": "whisper-large-v3",
  "maxDurationHours": 1
}
Shell
curl "$ENDPOINT_URL/v1/audio/transcriptions" \
  -F [email protected] \
  -F model=whisper-large-v3

pytorch-jupyter — PyTorch 2.5 + CUDA 12.4 JupyterLab

An interactive JupyterLab environment with PyTorch 2.5 and CUDA 12.4. The endpoint URL opens JupyterLab; the access token is printed to the job log at start-up. Files in the working directory live on the job's scratch storage and are deleted when the job ends — copy anything you want to keep to your own storage before stopping.

  • Min VRAM: 8 GB
  • Recommended: any; rtx4090 or a100-80 for model development
JSON
{
  "sku": "a100-80",
  "autoSelect": true,
  "template": "pytorch-jupyter",
  "maxDurationHours": 5
}

custom-docker — Custom Docker image

Runs any public container image that uses the NVIDIA runtime. Specify config.image and the ports to publish. Use this for your own inference servers, fine-tuning services or tools that are not covered by the other templates.

  • Min VRAM: any (pick a SKU that fits your workload)
JSON
{
  "offerId": "off_3h8v0rz2pa",
  "template": "custom-docker",
  "maxDurationHours": 12,
  "config": {
    "image": "ghcr.io/acme/embeddings-server:2.1.0",
    "ports": [8080],
    "env": { "MODEL_ID": "BAAI/bge-large-en-v1.5" }
  }
}

Images must be pullable without registry credentials. Read the Docker documentation on building GPU images and the NVIDIA Container Toolkit docs for runtime requirements.

batch — Batch job

Runs a container to completion and settles when it exits. You are charged for the seconds it ran; the rest of the deposit is refunded. Use it for evaluation runs, embedding a dataset, rendering or short fine-tuning jobs. The exit code is reported with the result.

  • Min VRAM: any
JSON
{
  "sku": "h100-pcie",
  "autoSelect": true,
  "template": "batch",
  "maxDurationHours": 3,
  "config": {
    "image": "ghcr.io/acme/eval-harness:0.9",
    "args": ["--tasks", "gsm8k,mmlu", "--output", "s3://acme-evals/run-112"],
    "env": { "AWS_ACCESS_KEY_ID": "...", "AWS_SECRET_ACCESS_KEY": "..." }
  }
}

경고

Environment variables are delivered only to the job's container, but that container runs on a third-party provider's machine. Use short-lived, narrowly scoped credentials. Read the Security model before sending secrets.