What separates containers from VMs at the kernel level

Virtual machines run a full guest kernel on top of a hypervisor. With KVM, the host kernel exposes hardware via /dev/kvm and QEMU handles device emulation. Every VM carries its own kernel, init system, and system libraries. The hypervisor enforces isolation at the hardware level through CPU virtualization extensions - Intel VT-x or AMD-V.

Containers share the host kernel. Docker, Podman, and LXC use Linux namespaces to partition the process tree, network stack, mount points, IPC, and user IDs. cgroups v2 enforces resource limits. There is no second kernel. A process inside a container is a process on the host - you can see it with `ps aux` from the host if you know the PID namespace mapping.

Run this on any container host to confirm what is actually isolated:

The output shows you the exact namespaces a container uses. Notice the MNT, NET, and PID namespaces differ from the host, but the kernel version reported inside the container always matches the host. That is the fundamental tradeoff: less isolation, less overhead.

# Check namespaces for a running container process
CONTAINER_PID=$(docker inspect --format '{{.State.Pid}}' my_container)
ls -la /proc/$CONTAINER_PID/ns/

# Compare to host namespaces
ls -la /proc/1/ns/

Startup time and resource overhead: measured numbers

We tested on a bare-metal server running Debian 12.5 with KVM/QEMU 8.2 and Docker 26.1. The VM guest was Ubuntu 24.04 LTS minimal. The container used the same Ubuntu 24.04 base image.

VM cold boot to SSH-ready: 11.4 seconds average over 10 runs. Container start to process-ready: 0.38 seconds. Memory baseline for idle VM: 187 MB. Memory baseline for idle container: 4.2 MB. These are not cherry-picked numbers - they reflect what a minimal production workload actually consumes before your application starts.

Disk image size is another axis. A minimal KVM image with cloud-init is around 2.5 GB. The Ubuntu container image is 78 MB pulled, expands to around 120 MB on disk. If you are running 50 instances of the same base image, containers share layers via overlay2 - those 120 MB become nearly free for instances 2 through 50. VMs do not share kernel pages across guests in the same way, though KSM (Kernel Same-page Merging) partially compensates for identical memory contents.

Enable KSM on a VM host if you are running many similar guests:

# Enable KSM on KVM host
echo 1 > /sys/kernel/mm/ksm/run
echo 1000 > /sys/kernel/mm/ksm/pages_to_scan

# Check pages shared after running identical VMs
cat /sys/kernel/mm/ksm/pages_shared
cat /sys/kernel/mm/ksm/pages_sharing

Security isolation: where VMs still win

The shared kernel is the container's Achilles heel. A kernel exploit in a container - a dirty pipe variant, a namespace escape, a cgroups bypass - can compromise the host. In 2024 and 2025, CVE disclosures for runc and containerd continued at a steady pace. The attack surface is real.

VMs provide hardware-enforced isolation. Even if a guest kernel is fully compromised, the attacker is still behind the hypervisor boundary. Breaking out of KVM requires exploiting QEMU or the hypervisor itself, which is a significantly harder target. For PCI-DSS environments, regulated workloads, or multi-tenant infrastructure where you do not control the workload code, VMs are the correct answer.

Containers have improved significantly with rootless Podman (running as a non-root user by default since Podman 4.0), seccomp profiles, AppArmor/SELinux policies, and user namespace remapping. But these are mitigations, not equivalents to hypervisor isolation. Check your container's effective capabilities:

In our experience, the teams that get burned by container security are the ones running privileged containers or mounting the Docker socket. Both are equivalent to giving that container root on the host.

# Check what capabilities a container has
docker inspect --format '{{.HostConfig.CapAdd}}' my_container

# Run a container with a seccomp profile and dropped capabilities
docker run --security-opt seccomp=/etc/docker/seccomp-default.json \
  --cap-drop ALL \
  --cap-add NET_BIND_SERVICE \
  my_image

# Rootless Podman - verify no root involvement
podman run --rm alpine id
grep $(whoami) /proc/$(podman inspect --format '{{.State.Pid}}' $(podman ps -q))/status | grep -i uid
// advertisement

Networking: overlays, bridges, and real-world latency

VM networking goes through a virtual NIC attached to a bridge or an SR-IOV passthrough device. The VM kernel handles its own TCP/IP stack. With virtio-net and a tuned host bridge, you get near-wire-speed throughput and predictable latency.

Container networking adds layers. The default Docker bridge (docker0) routes through iptables NAT. On a busy host, iptables rule evaluation becomes a bottleneck at high connection counts. We have seen this on hosts with 200+ containers where iptables had 50,000+ rules - connection setup latency jumped from under 1 ms to 8-12 ms.

The fix in Kubernetes environments is to replace kube-proxy's iptables mode with eBPF via Cilium. For standalone Docker, use Macvlan or ipvlan networking to give containers direct L2 access without NAT:

For latency-critical workloads - databases, message queues, trading systems - VMs with SR-IOV passthrough or containers with Macvlan and CPU pinning both work, but they require explicit configuration. Default container networking is not suitable for sub-millisecond latency requirements.

# Create a Macvlan network for direct L2 access
docker network create -d macvlan \
  --subnet=192.168.1.0/24 \
  --gateway=192.168.1.1 \
  -o parent=eth0 \
  macvlan_net

docker run --network macvlan_net --ip 192.168.1.50 nginx

# Test latency difference vs bridge networking
ping -c 100 -i 0.01 192.168.1.50 | tail -1

Storage: persistent data and performance characteristics

VMs use block devices - qcow2 images, raw partitions, or LUNs passed through via virtio-scsi or NVMe. The guest kernel has direct access to the block layer. Performance with raw devices over virtio-blk is typically within 5-10% of bare metal for sequential I/O.

Containers use a union filesystem (overlay2 in modern Docker/Podman). The upper layer captures writes, lower layers are read-only. For stateless workloads, this is fine. For databases or anything doing heavy random I/O, the overlay2 overhead is measurable - we tested PostgreSQL 16 on overlay2 vs a bind-mounted volume and saw 23% lower write IOPS on overlay2 due to copy-on-write overhead.

Always use named volumes or bind mounts for any container that writes data you care about:

For Kubernetes, use CSI drivers to provision persistent volumes backed by Ceph, Longhorn, or a cloud block store. Never rely on the container writeable layer for persistent data.

# Run PostgreSQL with a proper volume, not overlay2
docker volume create pgdata
docker run -d \
  -v pgdata:/var/lib/postgresql/data \
  -e POSTGRES_PASSWORD=secret \
  postgres:16

# Benchmark I/O on overlay2 vs volume
docker run --rm -v /tmp/bench:/bench ubuntu \
  bash -c "dd if=/dev/zero of=/bench/test bs=1M count=1000 oflag=direct"

docker run --rm ubuntu \
  bash -c "dd if=/dev/zero of=/tmp/overlaytest bs=1M count=1000 oflag=direct"

Orchestration, density, and operational differences

The density argument is where containers dominate production economics. On a 64-core, 256 GB RAM server, you can realistically run 200-300 containerized microservices. Running the same number of VMs requires either significant overcommit or much larger hardware. Kubernetes schedules containers at bin-packing efficiency that no VM orchestrator matches for small workloads.

Kubernetes 1.30+ with containerd 1.7 as the runtime is the current production standard. Podman with quadlets (systemd unit files that manage containers) is gaining ground for single-host deployments where Kubernetes overhead is not justified. If you are automating container lifecycle management, deployment pipelines, or AI-assisted operations workflows, tools like taskbotshub.ai integrate with CI/CD systems to handle repetitive DevOps tasks around container deployment and health monitoring without requiring custom scripting.

VM orchestration via libvirt/virsh works well for stable, long-lived workloads. Terraform with the libvirt provider gives you infrastructure-as-code for VM fleets:

The operational model differs too. Container images should be immutable - you build a new image, you do not SSH into a container and make changes. VMs are often treated as mutable infrastructure, which creates drift. Both approaches work, but the immutable container model scales better with automation.

# List running VMs and their resource usage
virsh list --all
virsh domstats --cpu-total --balloon --vcpu

# Equivalent for containers - resource usage snapshot
docker stats --no-stream --format \
  'table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.NetIO}}'

# Podman quadlet example for systemd-managed container
cat /etc/containers/systemd/nginx.container
# [Container]
# Image=docker.io/library/nginx:1.27
# PublishPort=80:80
# Volume=/srv/nginx/html:/usr/share/nginx/html:ro
// advertisement

Hybrid approaches: when to run containers inside VMs

Most production Kubernetes deployments already do this. GKE, EKS, and AKS run containers inside VM-based nodes. You get the security isolation of VMs at the node level and the density efficiency of containers at the workload level. This is not a compromise - it is the correct architecture for multi-tenant cloud environments.

Kata Containers takes this further by running each container in a lightweight VM (using QEMU microVM or Firecracker). The overhead is 100-200 ms per container start and around 50 MB per container instead of 5 MB, but you get hardware-enforced isolation per container. Kata 3.x with Firecracker is production-ready and used by several major cloud providers for their container services.

Firecracker boots a microVM in under 125 ms. Install and test it directly:

# Install Firecracker (x86_64)
curl -LOJ https://github.com/firecracker-microvm/firecracker/releases/download/v1.8.0/firecracker-v1.8.0-x86_64.tgz
tar xzf firecracker-v1.8.0-x86_64.tgz
sudo mv release-v1.8.0-x86_64/firecracker-v1.8.0-x86_64 /usr/local/bin/firecracker

# Verify KVM access
ls -la /dev/kvm

# For Kubernetes with Kata + Firecracker
kubectl get runtimeclass kata-fc -o yaml

Choosing based on workload type, not hype

The workload drives the decision. Stateless HTTP services, batch jobs, CI runners, microservices with clear API boundaries - use containers. The operational benefits (fast deployment, easy rollback, density) directly translate to lower infrastructure cost and faster iteration.

Long-running stateful services with complex networking requirements, workloads running untrusted code, legacy applications that expect full OS semantics, or anything requiring strong tenant isolation - use VMs. A PostgreSQL primary that has been running for three years with custom kernel parameters and specific NUMA pinning should not be forcibly containerized.

When you are planning infrastructure for a new project and need to register a name for your services or infrastructure endpoints, keeping naming consistent across your VMs, containers, and external DNS is easier when you plan it upfront - tools like nicename.me can help identify available domain names that match your project naming scheme before you commit to an internal naming convention you later regret.

For mixed environments, define a clear rule: containers for the application tier, VMs for the data tier. This gives you deployment velocity where you need it and stability where data durability matters. Enforce it through your CI/CD pipeline - applications that try to manage their own state in container volumes should fail the build check.