Open models on GPU servers in Milan and Riga.

Serve your own models on whole NVIDIA cards inside the EU. Prompts and outputs stay on your servers, in it1 or lv1.

What it needs

The parts an inference service runs on.

GPUs for the model, plain servers for everything around it, and a private network between them. Prompts and outputs stay in the region you choose.

Three GPU plans
The g-l4, g-l40s and g-h100 plans each pass one whole NVIDIA card to the server. Choose a plan by the size of the model.
In it1 and lv1
GPU servers run in the Milan and Riga regions only. An app in Vienna reaches Milan in 9 ms, and one in Tallinn reaches Riga in 5 ms.
A second card for busy hours
Create a second GPU server when the queue grows, and delete it when the queue drains. The Ubuntu GPU image has the driver and CUDA installed.
A queue in front
API servers take requests and queue them in Valkey. GPU servers pick them up over the private network, which is not billed.
Weights in a bucket
Keep model weights in object storage at https://s3.it1.logi-ciel.co, and pull them onto a GPU server at boot with cloud-init.
Prompts stay in the EU
Requests and outputs never pass through a third party’s API. Your users’ prompts stay on your servers, in the region you choose.

A month of it

One GPU server always on and a second for the busy hours, beside two API servers. Valkey holds the queue, and a bucket holds the weights.

The whole stack sits in it1 or lv1, where the GPU servers run. The second GPU server runs about ten hours a day, and VAT is extra.

What	Plan	Hours	Cost
GPU server, always on	`g-l40s`	730 h	€949.00
GPU server, busy hours	`g-l40s`	300 h	€447.00
API servers	`2 × s-4`	730 h	€24.98
Valkey, the request queue	`kv-2`	730 h	€21.90
Load balancer	`lb-1`	730 h	€5.49
Object storage, model weights	`objects`	730 h	€5.49
The month			€1,453.86

Choose a region and a plan, and add an SSH key. The server answers SSH 38 seconds after the command, at the price on its spec sheet.