跳至正文
MZGrid

此页面仅提供英文版本。 以英文版本为准。

Quickstart

Deploy an OpenAI-compatible Llama 3.1 8B Instruct endpoint on MZGrid in a few minutes, from the website or from the TypeScript SDK.

This guide deploys the llama-3.1-8b template — Llama 3.1 8B Instruct served by vLLM behind an OpenAI-compatible API — on a 24 GB GPU and sends it a chat completion. Total spend for a two-hour reservation on an RTX 4090 offer at 155 MZG/h is a 310 MZG deposit, and you are only charged for the seconds the endpoint actually runs.

Before you begin

You need:

  • A browser wallet (any injected EIP-1193 wallet) connected to Base (chain id 8453).
  • Enough MZG for the deposit — for this guide, about 310 MZG.
  • A small amount of ETH on Base to pay gas for the funding transaction.
  • Access to the Llama 3.1 weights. The template pulls meta-llama/Llama-3.1-8B-Instruct from Hugging Face, which is a gated model: accept the license on its model page and create a read token.

Option A — Deploy from the website

  1. Open the marketplace and filter by VRAM ≥ 24 GB. RTX 4090, RTX 3090, RTX 5090 and L40S offers all fit an 8B model.
  2. Choose an offer. Sort by reliability if you want the most consistent providers, or by price for the cheapest. Check that the offer is verified.
  3. On the offer page select the Llama 3.1 8B Instruct (vLLM) template and set the duration to 2 hours. The calculator shows the deposit: 155 × 2 = 310 MZG.
  4. Under Environment, add HF_TOKEN with your Hugging Face read token.
  5. Click Deploy. Connect your wallet; if it is on another network, approve the switch to Base.
  6. Sign the permit (a gas-free signature that authorises the escrow to pull exactly the deposit) and confirm the createJobWithPermit transaction. If your wallet does not support typed-data signatures, the app falls back to an approve transaction followed by createJob.
  7. Watch the status move from funded → starting → running. Model download and warm-up usually take two to four minutes on a 24 GB card.
  8. Copy the endpoint URL from the job panel. You can also follow it from your dashboard.

Option B — Deploy with the SDK

Install the SDK and viem:

Shell
npm install @mzgrid/sdk viem

Create an API key in your dashboard (or via POST /v1/keys) and export it together with the funding wallet's private key and your Hugging Face token:

Shell
export MZG_API_KEY="mzg_live_..."
export WALLET_PRIVATE_KEY="0x..."
export HF_TOKEN="hf_..."

Then run:

TypeScript
import { MZGridClient } from "@mzgrid/sdk";
import { createWalletClient, http } from "viem";
import { privateKeyToAccount } from "viem/accounts";
import { base } from "viem/chains";

const walletClient = createWalletClient({
  account: privateKeyToAccount(process.env.WALLET_PRIVATE_KEY as `0x${string}`),
  chain: base,
  transport: http(),
});

const client = new MZGridClient({
  apiUrl: "https://api.mzgrid.io",
  apiKey: process.env.MZG_API_KEY,
});

// 1. Create the job. The scheduler picks the best-scoring RTX 4090 offer.
const created = await client.jobs.create({
  sku: "rtx4090",
  autoSelect: true,
  template: "llama-3.1-8b",
  maxDurationHours: 2,
  config: { env: { HF_TOKEN: process.env.HF_TOKEN ?? "" } },
});
console.log("deposit:", created.job.deposit, "MZG");

// 2. Fund the escrow (permit + createJobWithPermit, or approve + createJob).
await client.jobs.fund(created, walletClient);

// 3. Wait until the container reports its endpoint.
const job = await client.jobs.waitForEndpoint(created.job.id, { timeoutMs: 10 * 60_000 });
console.log("endpoint:", job.endpointUrl);

Send a request

The endpoint speaks the OpenAI Chat Completions API under /v1:

Shell
curl "$ENDPOINT_URL/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [{"role": "user", "content": "Explain per-second GPU billing in one sentence."}],
    "max_tokens": 120
  }'

Any OpenAI-compatible client works — point its base URL at ENDPOINT_URL + "/v1". See Using the API from Python for an example with the openai package.

Stop and get refunded

When you are done, stop the job from the dashboard or with client.jobs.stop(jobId). The coordinator settles with the seconds used so far. If you stopped after 45 minutes:

MZG
Deposit310.00
Charged (2,700 s × 155 / 3,600)116.25
Refunded to you193.75

Next steps