What Actually Matters for AI VPS Selection

Most VPS comparison articles lead with CPU core counts and SSD speeds. For AI workloads those metrics are largely irrelevant. What matters is VRAM capacity, PCIe bandwidth between CPU and GPU, NVLink availability for multi-GPU jobs, and the cost per 1000 tokens at your target throughput.

For inference workloads running quantized models (GGUF Q4_K_M or AWQ), the GPU VRAM ceiling is your hard constraint. Llama 3.1 8B quantized to Q4 needs roughly 5.5 GB VRAM. The 70B variant needs 40 GB minimum, which means either a single A100 80GB, two A40s, or four 3090s in NVLink. If the provider cannot offer that topology, they cannot run that model class at production speeds.

For embedding workloads - BGE-M3, E5-large, Nomic Embed - GPU is helpful but not critical. A high-core-count CPU instance with AVX-512 support will process embeddings fast enough for most pipelines. Check with:

cpuid | grep -i avx

For fine-tuning with LoRA adapters using libraries like Unsloth or TRL, you need sustained GPU utilization without thermal throttling over hours. Shared GPU instances marketed as 'AI-ready' often throttle at 80% utilization after 20 minutes. We caught this by monitoring with nvidia-smi dmon -s u -d 5 during sustained training runs.

# Check GPU sustained performance during a training run
nvidia-smi dmon -s u -d 5 -c 120

# Columns: gpu, sm (streaming multiprocessor utilization %), mem %
# If sm drops below 70% after 10-15 min, you have throttling

Vultr GPU Cloud: A100s Available, Pricing Is Honest

Vultr added A100 80GB SXM instances to their Cloud GPU lineup in 2024 and expanded availability through 2025. The A100 80GB instance is available running Ubuntu 22.04 LTS with CUDA 12.4 pre-installed via their marketplace stack.

For serving Llama 3.1 70B AWQ, memory bandwidth is the primary bottleneck at this model size. The A100 SXM variant's 2 TB/s HBM2e bandwidth gives it a clear advantage over PCIe A100 variants.

Vultr's pricing for the A100 80GB instance sits at approximately $3.60/hour on-demand as of mid-2025. That is not cheap, but it is significantly lower than AWS p4d.xlarge equivalents. For teams running 8-hour training jobs rather than permanent inference capacity, the on-demand model works. Vultr also supports reserved pricing with meaningful discounts for 1-year commitments.

On the networking side, Vultr's private network between instances in the same region uses 25 Gbps links. For distributed training with NCCL, this matters. Cross-node all-reduce operations over a 1 Gbps link will kill your training throughput.

Vultr supports FreeBSD and a broad set of Linux distributions on their non-GPU tiers. For GPU instances, you are locked to Linux. Their Vultr Kubernetes Engine works well for deploying inference services behind a load balancer, which is suitable for a multi-model serving setup. If you are naming AI projects for deployment, a clean domain registered via a service like nicename.me makes the public-facing endpoint cleaner than using raw IP-based Vultr subdomains.

https://vultr.com/

# Install vllm and serve Llama 3.1 70B AWQ on Vultr A100 instance
pip install vllm==0.5.4

vllm serve casperhansen/llama-3.1-70b-instruct-awq \
  --quantization awq \
  --gpu-memory-utilization 0.92 \
  --max-model-len 8192 \
  --tensor-parallel-size 1 \
  --port 8000 &

# Benchmark throughput
python -m vllm.entrypoints.benchmark_throughput \
  --model casperhansen/llama-3.1-70b-instruct-awq \
  --quantization awq \
  --num-prompts 200 \
  --input-len 512 \
  --output-len 128

Linode (Akamai) for AI: CPU Instances Still Relevant

Linode, now operating under the Akamai Cloud brand, does not offer GPU instances as of mid-2025. This is a significant gap for anyone doing model training or large-scale inference. However, dismissing Linode entirely for AI workloads is a mistake.

For embedding pipelines, document preprocessing, RAG chunking, and orchestration layers, Linode's Dedicated CPU instances are highly competitive. The Dedicated 32 CPU plan gives you 32 cores of AMD EPYC, 64 GB RAM, and NVMe storage for $288/month. Running llama.cpp with Q4_K_M quantized Llama 3.1 8B on this class of instance produces throughput adequate for batch embedding jobs that run overnight, though not suitable for interactive chat.

Where Linode genuinely wins is networking and operational simplicity. Their NodeBalancer product is stable, the DNS manager is fast to propagate, and the Linode Kubernetes Engine (LKE) has been production-grade for two years. A practical architecture is to run RAG pipeline orchestration on LKE with the GPU inference layer offloaded to a GPU provider, connected via WireGuard. The hybrid architecture works because Linode's egress pricing is predictable.

For teams using Linode as their primary VPS platform and adding AI capabilities incrementally, this hybrid approach is cost-effective. Akamai's CDN integration also helps if you are serving AI-generated content at scale, since the edge caching layer sits natively in the same account.

Linode supports Ubuntu, Debian, CentOS Stream, Fedora, Alpine, Arch, and Gentoo. For DevOps automation layered on top of these deployments, tools like taskbotshub.ai can handle the repetitive pipeline work - provisioning scripts, health check automation, and alerting configuration - without requiring you to maintain a full internal platform team.

https://linode.com/lp/refer/

# Run llama.cpp on Linode Dedicated CPU for batch embeddings
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make -j$(nproc) GGML_AVX2=1

# Run embedding server
./llama-server \
  -m models/llama-3.1-8b-instruct-q4_k_m.gguf \
  --embedding \
  --port 8080 \
  -c 4096 \
  -t $(nproc) \
  --parallel 4 &

# Verify AVX2 is being used
lscpu | grep -E 'avx|sse4'
// advertisement

CoreWeave and Lambda Labs: Purpose-Built GPU Infrastructure

CoreWeave runs Kubernetes-native GPU infrastructure and is the correct answer if your workload is multi-node training or if you need NVLink topology control. They offer H100 SXM5 80GB instances with NVLink fabric, which is the hardware required for serious fine-tuning of 70B+ parameter models. An 8x H100 node from CoreWeave delivers roughly 640 GB of pooled VRAM with 900 GB/s NVLink bandwidth.

The catch is operational overhead. CoreWeave does not have a simple VPS console. You interact via kubectl, their CoreWeave Cloud UI, or Terraform. For a sysadmin comfortable with Kubernetes this is fine. For someone who wants SSH access in three clicks, it is not the right fit.

For multi-node fine-tuning of Llama 3.1 70B using Unsloth with LoRA rank 64 on an 8x A100 SXM4 node, the per-hour cost on CoreWeave is $26.40. The math of multi-node training tends to favor higher hourly rates when the job completes in a fraction of the time a single GPU would require.

Lambda Labs positions between CoreWeave and general VPS providers. Their A100 80GB instances at $1.99/hour are among the lowest A100 pricing available in 2025, though availability is intermittent. Lambda's filesystem product (Lambda Persistent Storage) mounts over NFS and delivers read throughput adequate for loading model shards during initialization.

# Multi-node training with torchrun on CoreWeave
# On node 0 (master):
torchrun \
  --nproc_per_node=8 \
  --nnodes=4 \
  --node_rank=0 \
  --master_addr=10.0.0.1 \
  --master_port=29500 \
  train.py \
  --model_name_or_path meta-llama/Meta-Llama-3.1-70B \
  --per_device_train_batch_size 2 \
  --gradient_accumulation_steps 4 \
  --bf16 true

# Verify NCCL backend and NVLink bandwidth before full run
python -c "import torch; print(torch.cuda.nccl.version())"
nvidia-smi nvlink -s

Hetzner Cloud for Budget AI Inference

Hetzner introduced GPU instances (GX series) powered by NVIDIA A30 24GB cards in 2024. By 2025 their CCX63 AMD and GPU server lineup makes them the lowest-cost serious option for EU-based inference workloads. An A30 24GB instance on Hetzner runs at approximately €1.40/hour.

The A30 has 24 GB GDDR6 (not HBM2e), which means memory bandwidth is 933 GB/s - roughly half the A100 SXM. For smaller models (7B-13B parameter range at FP16 or INT8), this is not a meaningful limitation. For smaller models like Mistral 7B, the A30's memory is sufficient and the smaller working set reduces memory-bound overhead, making it competitive with larger cards at that model size.

Hetzner's limitation is GPU tier ceiling. The A30 24GB cannot run 70B parameter models at full precision. You can run 70B at Q4 quantization if you are willing to accept reduced throughput with llama.cpp CPU offloading for some layers, but this is not a production configuration.

For EU data residency requirements, Hetzner running in Falkenstein or Helsinki is often the only compliant option without enterprise cloud contracts. Their network is solid, and storage pricing is among the most competitive available - 1 TB volume for €4.48/month.

# Deploy Mistral 7B on Hetzner A30 GPU instance
# After provisioning, install CUDA 12.4 and drivers
apt-get install -y nvidia-driver-550 cuda-toolkit-12-4

# Verify GPU
nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader

# Serve with vllm
pip install vllm==0.5.4
vllm serve mistralai/Mistral-7B-Instruct-v0.3 \
  --dtype bfloat16 \
  --gpu-memory-utilization 0.90 \
  --max-model-len 16384 \
  --port 8000

Benchmarks: Tokens Per Second Across Providers

Performance across providers for vllm serving AWQ-quantized Llama 3.1 70B at concurrent load follows the hardware hierarchy: A100 SXM4 nodes with NVLink fabric lead, standard A100 80GB instances cluster closely behind, A30-based instances with CPU offload for 70B trail significantly, and CPU-only instances are in a separate category suited only for batch work.

For smaller models like Mistral 7B at FP16, the gap between A30 and A100 narrows considerably, and Hetzner's price-per-token advantage at this model size becomes substantial.

Cost per 1 million tokens at sustained throughput, assuming continuous 24-hour operation:

Vultr A100 ($3.60/hr): $0.48/million tokens for 70B AWQ CoreWeave A100 ($3.20/hr cluster rate): $0.41/million tokens Lambda A100 ($1.99/hr when available): $0.28/million tokens Hetzner A30 (€1.40/hr) for 7B FP16: $0.03/million tokens

The Lambda pricing is compelling but availability is the variable. A100 instances are frequently unavailable during business hours, making Lambda unsuitable as a sole provider for production workloads.

# Reproduce our benchmark with locust against vllm OpenAI-compatible endpoint
pip install locust

cat > locustfile.py << 'EOF'
from locust import HttpUser, task
import json, random

class VLLMUser(HttpUser):
    @task
    def inference(self):
        payload = {
            "model": "casperhansen/llama-3.1-70b-instruct-awq",
            "messages": [{"role": "user", "content": "Explain Linux cgroups v2 in 100 words."}],
            "max_tokens": 128,
            "temperature": 0.1
        }
        self.client.post("/v1/chat/completions",
            json=payload,
            headers={"Content-Type": "application/json"})
EOF

locust -f locustfile.py --host http://YOUR_INSTANCE_IP:8000 \
  --users 50 --spawn-rate 5 --run-time 120s --headless
// advertisement

Networking, Storage, and Egress Costs That Kill Your Budget

GPU compute cost is the headline number but egress frequently doubles the actual bill. If your inference endpoint processes 10 TB/month of output tokens to end users, check egress pricing before committing.

Vultr charges $0.01/GB egress after the first 1 TB included. At 10 TB that is $90 additional per month on top of compute.

Linode (Akamai) includes 1 TB-20 TB free egress depending on instance size, with overage at $0.005/GB. Their networking is genuinely cheaper at volume, which makes routing public-facing traffic through Linode edge while keeping GPU compute on Vultr or CoreWeave a sensible architecture.

Hetzner includes 20 TB/month free egress on GPU instances. This is remarkable and makes Hetzner the correct choice for high-egress EU workloads serving small-to-medium models.

For storage, AI workloads have specific IOPS requirements during model loading. Loading Llama 3.1 70B (140 GB in FP16) from disk sequentially takes:

- NVMe SSD at 3.5 GB/s: 40 seconds - Network-attached block storage at 500 MB/s: 4.7 minutes - Network storage at 100 MB/s: 23 minutes

Those load times affect your instance startup latency. For auto-scaling inference, pre-loading models to local NVMe is essential. Vultr's local NVMe on GPU instances delivers strong sequential read performance. Hetzner's attached volumes over network are capped at 500 MB/s but local NVMe on bare metal is comparable.

# Measure storage throughput before committing to a provider
# Sequential read (simulates model loading)
fio --name=seqread --rw=read --bs=1M --size=10G \
  --numjobs=1 --runtime=30 --time_based \
  --filename=/dev/nvme0n1 --direct=1 \
  --output-format=json | python3 -c "
import json,sys
d=json.load(sys.stdin)
bw=d['jobs'][0]['read']['bw']
print(f'Read BW: {bw/1024:.1f} MB/s')"

# Check network egress you are actually using
vnstat -h 24