Back to Catalog
Create Endpoint

This model is from our Model Catalog, and comes with pre-configured recipes. Deployment has been verified by Hugging Face.

/
$2.50 / h
per running replica

General purpose Recipe

Balances answer speed and capacity, so one endpoint copes with varied requests.

Best for: Mixed everyday traffic. Trade-off: A specialised recipe beats it on that specialty, on the same hardware..
Input throughput
1,888.79 tok / s
Output throughput
233.53 tok / s
Request throughput
0.96 req / s
Cost per 1M tokens
$0.33
Time to first token
1,866 ms
Inter-token latency
40.59 ms
Generation speed
33.83 tok / s
Response time
16,289 ms

Measured at 8 concurrent requests of about 2,048 input and 256 output tokens — the heaviest measured load that still met every target.

Nvidia A100
1x GPU · 80 GB 11x vCPUs · 145 GB
$2.5 / h
Catalog Recipe
Pre-selected hardware for the current recipe.
  • Only you can access your endpoint, using a Hugging Face Token generated from your personal account.
Number of replicas
More options
Daily Autoscaling Schedule
Autoscaling Strategy

This Catalog Recipe comes with a pre-configured vLLM engine.

Engine Container URL optional

Default Env values come from the Catalog Recipe and can't be edited during Endpoint creation.

Default Env
Secret Env
VPC Config