Back to Catalog
Create Endpoint

This model is from our Model Catalog, and comes with pre-configured recipes. Deployment has been verified by Hugging Face.

/
$20 / h
per running replica

General purpose Recipe

Balances answer speed and capacity, so one endpoint copes with varied requests.

Best for: Mixed everyday traffic. Trade-off: A specialised recipe beats it on that specialty, on the same hardware..
Input throughput
12,462.17 tok / s
Output throughput
1,556.62 tok / s
Request throughput
10.12 req / s
Cost per 1M tokens
$0.40
Time to first token
700 ms
Inter-token latency
32.78 ms
Generation speed
36.9 tok / s
Response time
14,960 ms

Measured at 64 concurrent requests of about 2,048 input and 256 output tokens — the heaviest measured load that still met every target.

Nvidia H200
4x GPUs · 564 GB 92x vCPUs · 1024 GB
$20 / h
Catalog Recipe
Pre-selected hardware for the current recipe.
  • Only you can access your endpoint, using a Hugging Face Token generated from your personal account.
Number of replicas
More options
Daily Autoscaling Schedule
Autoscaling Strategy

This Catalog Recipe comes with a pre-configured vLLM engine.

Engine Container URL optional

Default Env values come from the Catalog Recipe and can't be edited during Endpoint creation.

Default Env
Secret Env
VPC Config