GPU 4x Nvidia H200 vllm $20 / hr CUSTOM Define your own configuration. Custom

Contact us if you'd like to request a custom solution or instance type.

Amazon Web Services

Microsoft Azure

Google Cloud Platform

CPU GPU INF2

Nvidia L40S

4x GPUs · 192 GB 47x vCPUs · 380 GB

$8.3 / h

suggested Best performance/price ratio for selected model.

Private Public Authenticated

Only you can access your endpoint, using a Hugging Face Token generated from your personal account.

Automatic Scale-to-Zero

Endpoints scaled to 0 replicas are not billed. They may take some time to scale back up once they start receiving requests again.

Number of replicas

Automatically scale the number of replicas within Min and Max based on compute usage. Min is always 0 if Scale-To-Zero is active.

Min Max

More options

Autoscaling Strategy

Control what type of trigger will cause your Endpoint to scale up.

Hardware Usage Pending Requests

Hardware Utilization Threshold (%)

A scale up event will be triggered if the average hardware utilisation (%) exceeds this threshold for more than 20 seconds.

Inference Engine

Engine Container URL optional

Max Batched Tokens

Maximum number of tokens to be processed in a single iteration. If left unspecified by the user, vLLM will be set it based on the usage context.

optional

Max Sequences

Maximum number of sequences to be processed in a single iteration. If left unspecified by the user, vLLM will be set it based on the usage context.

optional

Tensor Parallel Size

Number of tensor parallel groups for distributed inference

optional

Data Parallel Size

Number of data parallel groups for distributed inference

optional

KV Cache Data Type

KV cache data type for memory optimization

optional

Server Arguments

Arguments passed to the engine server entrypoint.

optional

Server Command

Command executed as engine server entrypoint.

optional

Default Env

Environment variables that will be provided to your container during deployment.

Secret Env

Same as Default, but people with access to this endpoint will not be able to read these values after creation.

VPC Config

Check to activate and configure AWS PrivateLink

Use AWS PrivateLink

DeepSeek-V4-Flash-Vision-Exp