How much does it cost to run a model server on Hugging Face Jobs?

An a10g-large GPU instance runs at $1.50/hour, and the service bills per second of hardware usage. Smaller or larger GPU flavors are available at different price points; you can check the full price list with the `hf jobs hardware` command.

What authentication is required to query the endpoint?

Every request must carry a Hugging Face token with read access to the job's namespace. The endpoint is gated and not publicly accessible; plain browser visits are rejected.

Can I run very large models on this service?

Yes. You can scale to much larger models by selecting beefier hardware flavors (such as h200x2 for a 122B parameter model) and using the `--tensor-parallel-size` flag to shard the model across multiple GPUs. Larger models require longer timeout values since they take longer to download and load.

Hugging Face BlogJun 25, 2026Jun 26, 2026, 06:01 JST1 min read

Hugging Face launches one-command AI model server

Send on LINE

Key takeaway

Hugging Face launched a simplified way to deploy private AI model servers using a single CLI command on its infrastructure, with OpenAI-compatible APIs and per-second billing.
This removes the need to provision and manage servers or Kubernetes, making it practical for developers to spin up models for testing and batch work without fixed infrastructure commitments.

3 Key Points

What happened
Hugging Face introduced a one-command way to launch a vLLM (a fast AI inference engine) server on its Jobs infrastructure, with OpenAI-compatible API access and per-second billing. Users authenticate with their Hugging Face token and can query the endpoint from anywhere.
Why it matters
Previously, setting up a private model server required provisioning and managing Kubernetes infrastructure. This removes that operational overhead, making it faster and cheaper to stand up models for testing, evaluation, or batch generation work without committing to fixed infrastructure costs.
What to watch
The service charges $1.50/hour for an a10g-large GPU instance, and users can scale to larger models by specifying bigger hardware flavors (like h200x2 for the 122B Qwen3.5 model). Jobs are billed per second, so explicit cancellation is cheaper than relying on the timeout safety net.

Ask the AI about this article →

Summaries like this, in your inbox every morning.

Get news like this every morning — free

FAQ

How much does it cost to run a model server on Hugging Face Jobs?: An a10g-large GPU instance runs at $1.50/hour, and the service bills per second of hardware usage. Smaller or larger GPU flavors are available at different price points; you can check the full price list with the `hf jobs hardware` command.
What authentication is required to query the endpoint?: Every request must carry a Hugging Face token with read access to the job's namespace. The endpoint is gated and not publicly accessible; plain browser visits are rejected.
Can I run very large models on this service?: Yes. You can scale to much larger models by selecting beefier hardware flavors (such as h200x2 for a 122B parameter model) and using the `--tensor-parallel-size` flag to shard the model across multiple GPUs. Larger models require longer timeout values since they take longer to download and load.

Hugging Face BlogRead Original Article

Get AI news like this every morning

AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.

Subscribe free →

Free · takes 30 seconds · unsubscribe anytime

Ask AI

Ask AI anything about this article. Q&As are published on this page for other readers too.

Next articlePatronus AI raises $50 million for AI agent testing