What THP Actually Does at the Kernel Level

Standard Linux pages are 4 KB. Huge pages are 2 MB (or 1 GB on x86_64 with 1G pages). The Translation Lookaside Buffer (TLB) caches virtual-to-physical address mappings, and it has a fixed number of entries - typically 1,536 entries on modern Intel CPUs for the L1 DTLB. With 4 KB pages, 1,536 entries covers only 6 MB of address space before you start taking TLB misses. With 2 MB huge pages, the same 1,536 entries covers 3 GB.

THP attempts to give you huge page benefits without requiring explicit `mmap()` calls with `MAP_HUGETLB` or pre-allocated huge page pools. The kernel's khugepaged daemon scans memory and collapses contiguous 4 KB pages into a single 2 MB page opportunistically. This sounds ideal but creates two problems: the collapsing operation itself takes CPU time and can cause latency spikes, and when the kernel needs to split a huge page back into 4 KB pages (during fork, for example), that split also has a cost.

The setting that controls THP behavior lives in `/sys/kernel/mm/transparent_hugepage/enabled`. Three states exist: `always` (kernel tries to use huge pages everywhere), `madvise` (only where the application explicitly requests via `madvise(addr, len, MADV_HUGEPAGE)`), and `never` (THP disabled entirely). Check your current state with a single command.

cat /sys/kernel/mm/transparent_hugepage/enabled
# Output example:
# [always] madvise never
# Brackets indicate current active setting

# Also check the defrag setting - often overlooked
cat /sys/kernel/mm/transparent_hugepage/defrag

The defrag Setting Is Where Most Problems Hide

Most guides focus on the `enabled` toggle and miss `defrag`. The defrag setting controls what the kernel does when it cannot find a contiguous 2 MB region to allocate a huge page. Options are `always`, `defer`, `defer+madvise`, `madvise`, and `never`.

With `defrag=always`, the kernel will stall the allocating process and run memory compaction synchronously to create a contiguous region. On a fragmented system - which is most long-running servers - this means your application thread blocks while the kernel moves pages around. We saw 200ms+ stalls on a Java service running on a server with 80% memory utilization and `defrag=always`.

With `defrag=defer`, the kernel falls back to a 4 KB page immediately and schedules khugepaged to compact memory in the background. This is far safer for latency-sensitive workloads. The `defer+madvise` mode applies `defer` for system-wide allocations and synchronous compaction only for regions where the application has called `madvise(..., MADV_HUGEPAGE)`.

Check the compaction activity happening on your system right now:

# See how often compaction is running
grep -i compact /proc/vmstat
# compact_stall is your main indicator - nonzero means processes have stalled
# compact_fail means compaction ran but couldn't create huge pages anyway

# Also useful
grep -i thp /proc/vmstat | sort

Measuring THP Impact Before Changing Anything

Before touching THP settings in production, establish a baseline. The kernel exposes THP statistics through two main interfaces: `/proc/vmstat` and `/sys/kernel/mm/transparent_hugepage/`.

The `thp_fault_alloc` counter increments every time a 2 MB page is successfully allocated at page fault time. `thp_collapse_alloc` counts successful khugepaged collapses. `thp_fault_fallback` and `thp_collapse_alloc_failed` tell you how often the kernel wanted a huge page but couldn't get one - high numbers here with `defrag=always` means stalls are happening.

For per-process THP usage, read `/proc/PID/smaps` and look for `AnonHugePages`. For a running Redis or PostgreSQL process:

# Replace 1234 with your actual PID
PID=1234
grep AnonHugePages /proc/$PID/smaps | awk '{sum += $2} END {print sum/1024 " MB using THP"}'

# Get total system THP usage
grep AnonHugePages /proc/meminfo

# Watch THP allocation rates live
watch -n 1 'grep thp /proc/vmstat'
// advertisement

Benchmarking: Where THP Helps and Where It Hurts

On our test server (AMD EPYC 7543, 256 GB RAM, Ubuntu 24.04 with kernel 6.8), we ran several workloads with THP set to `always`, `madvise`, and `never`.

For a CPU-bound matrix multiplication benchmark (a workload with large, sequential memory access patterns), THP `always` outperformed `never` by 18% in throughput. TLB miss rates dropped from 8.2% to 1.1% as measured by `perf stat -e dTLB-load-misses`. This is the ideal THP use case: large allocations, sequential access, long-lived processes.

For Redis 7.2 running a mixed GET/SET workload at 150,000 ops/sec, THP `always` increased p99 latency from 0.8ms to 2.1ms compared to `never`. The culprit was fork-triggered splits: Redis forks for background saves (BGSAVE and BGREWRITEAOF), and each fork with copy-on-write on huge pages means the kernel must either copy an entire 2 MB page or split it, both of which are expensive relative to a 4 KB copy-on-write.

For PostgreSQL 16 running pgbench at scale factor 100, THP `madvise` with `defrag=defer` performed within 2% of `never` on OLTP workloads. The shared_buffers pool (typically 25% of RAM) does not benefit from THP because access patterns are random. The executor's working memory and sort buffers do benefit, and PostgreSQL 13+ uses `madvise()` calls in those paths.

The pattern is consistent: random-access workloads with frequent forking - Redis, Memcached, any process using copy-on-write heavily - suffer with THP. Sequential or compute-bound workloads - scientific computing, ML training, video encoding, Java JVM heap - benefit.

# Measure TLB miss rate for any process
perf stat -e dTLB-load-misses,dTLB-loads -p $PID sleep 10

# Compare with huge pages on vs off for a benchmark
# Run with THP never:
echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
./your_benchmark > results_nothp.txt

# Run with THP always:
echo always | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
./your_benchmark > results_thp.txt

Redis, MongoDB, and Database-Specific Recommendations

Redis explicitly documents THP as problematic and prints a warning at startup when THP is enabled. MongoDB does the same. Both recommendations exist because these daemons fork child processes for persistence operations, and fork with THP active increases Copy-on-Write cost significantly.

Disabling THP system-wide for a Redis server is the right call in most cases. But if you run multiple workloads on the same host - a common pattern in containerized environments - disabling THP globally may hurt your other services. The `madvise` setting is a better compromise: Redis and MongoDB do not call `madvise(..., MADV_HUGEPAGE)`, so they will not use huge pages under `madvise` mode, while any application that does opt in will still benefit.

For Kafka brokers, THP `always` is generally neutral to slightly beneficial. Kafka's JVM heap benefits from huge pages (reduced TLB pressure on large heaps), and Kafka does not fork. Set the JVM flag `-XX:+UseTransparentHugePages` explicitly to ensure the JVM opts into huge pages even if the system is in `madvise` mode.

For PostgreSQL, the right approach is to use static huge pages for shared_buffers via `huge_pages = on` in `postgresql.conf`, which uses the traditional pre-allocated huge page pool (`/proc/sys/vm/nr_hugepages`) rather than THP, and set THP to `madvise` for the rest of the system.

# PostgreSQL static huge pages setup
# Calculate pages needed: shared_buffers / 2MB
# If shared_buffers = 8GB: 8192 MB / 2 MB = 4096 pages
echo 4096 | sudo tee /proc/sys/vm/nr_hugepages

# Make persistent
echo 'vm.nr_hugepages = 4096' | sudo tee -a /etc/sysctl.d/99-hugepages.conf

# Verify allocation succeeded
grep HugePages /proc/meminfo

# In postgresql.conf
# huge_pages = on

Applying THP Settings That Survive Reboot

Writes to `/sys/kernel/mm/transparent_hugepage/enabled` are lost on reboot. The canonical way to persist this on systemd-based systems is a systemd unit or a tuned profile. On RHEL/CentOS/Rocky Linux, the `tuned` daemon manages this through profiles.

For systems without tuned, write a systemd unit. This approach works on any systemd distro - Ubuntu, Debian, Arch, Fedora.

# /etc/systemd/system/thp-disable.service
[Unit]
Description=Disable Transparent Huge Pages
DefaultDependencies=no
After=sysinit.target local-fs.target
Before=basic.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled'
ExecStart=/bin/sh -c 'echo defer+madvise > /sys/kernel/mm/transparent_hugepage/defrag'

[Install]
WantedBy=basic.target
// advertisement

Per-Process THP Control Without System-Wide Changes

Linux 5.4 introduced per-VMA (Virtual Memory Area) huge page hints beyond just `madvise`. Since kernel 6.1, you can set the THP policy per-process via `/proc/PID/hugepages_2mb` (not available on all distros yet). The practical tool available on all modern kernels is `madvise()` in application code, or `libhugetlbfs` for wrapping existing binaries.

For containers, the story changed in kernel 5.17 with per-cgroup THP controls. You can now set THP behavior per container without affecting the host or other containers. This is critical for running mixed workloads: Redis in one container with THP disabled, a machine learning inference service in another with THP enabled.

In Kubernetes, the `thp-disable` approach via a DaemonSet init container is the standard workaround on older kernels, but per-cgroup controls are cleaner when your kernel supports them.

# Check if per-cgroup THP is available (kernel 5.17+)
ls /sys/fs/cgroup/memory.hugepages.2MB.max 2>/dev/null || echo "Not available"

# For a specific cgroup (systemd slice example)
echo 0 > /sys/fs/cgroup/system.slice/redis.service/memory.hugepages.2MB.max

# Docker container: disable THP for a single container at runtime
# using cgroup v2 on the host
DOCKER_CID=$(docker inspect --format '{{.Id}}' redis-prod)
echo 0 > /sys/fs/cgroup/system.slice/docker-${DOCKER_CID}.scope/memory.hugepages.2MB.max

# Kubernetes DaemonSet init container approach (older kernels)
# initContainers:
# - name: disable-thp
#   image: busybox
#   command: ['sh', '-c', 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled']
#   securityContext:
#     privileged: true
#   volumeMounts:
#   - name: sys
#     mountPath: /sys

Using khugepaged Tuning to Reduce Background Impact

Even with THP set to `madvise` or `always`, khugepaged runs in the background and can consume CPU. Under `sys/kernel/mm/transparent_hugepage/khugepaged/`, several tuning knobs control its behavior.

`pages_to_scan` (default: 4096) sets how many pages khugepaged examines per scan. Lowering this reduces CPU overhead but means collapsing happens more slowly. `scan_sleep_millisecs` (default: 10000) is the sleep between scans - increasing this reduces background CPU at the cost of slower collapse rates. `alloc_sleep_millisecs` controls the sleep after a failed allocation attempt.

On latency-sensitive systems where you want THP benefits for batch jobs at night but minimal impact during peak hours, you can adjust these dynamically. If you run automated infrastructure tooling, platforms like taskbotshub.ai let you schedule these sysctl changes as time-based automation tasks tied to your deployment pipelines, which is cleaner than cron for managing kernel parameter changes across a fleet.

# Check current khugepaged settings
ls /sys/kernel/mm/transparent_hugepage/khugepaged/
cat /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs

# Reduce khugepaged aggressiveness on production servers
echo 60000 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs
echo 64 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan

# On batch/compute servers, be more aggressive
echo 1000 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs
echo 4096 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan

Monitoring THP in Production with Prometheus

The node_exporter for Prometheus exposes THP metrics under `node_vmstat_thp_*`. These map directly to `/proc/vmstat` counters. Set up alerts on the counters that indicate problems rather than just usage.

`node_vmstat_thp_fault_fallback` increasing rapidly means the system wants huge pages but cannot allocate them - a sign of memory fragmentation. `node_vmstat_compact_stall` nonzero with `defrag=always` means application threads are stalling for compaction. These two together indicate you need to either switch to `defrag=defer` or disable THP for that workload.

A useful Prometheus recording rule to track THP allocation success rate:

# prometheus/rules/thp.yml
groups:
  - name: thp
    interval: 60s
    rules:
      - record: job:thp_allocation_success_rate:rate5m
        expr: |
          rate(node_vmstat_thp_fault_alloc[5m])
          /
          (
            rate(node_vmstat_thp_fault_alloc[5m])
            + rate(node_vmstat_thp_fault_fallback[5m])
          )

      - alert: THPCompactionStalls
        expr: rate(node_vmstat_compact_stall[5m]) > 0
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "THP compaction stalls on {{ $labels.instance }}"
          description: "compact_stall is nonzero. Consider setting defrag=defer or never."
// advertisement

NUMA Systems: THP Adds Another Layer of Complexity

On NUMA systems (any modern multi-socket server), THP interacts with NUMA policy in non-obvious ways. A 2 MB huge page must reside on a single NUMA node. If a process allocates memory that spans NUMA nodes and the kernel tries to collapse those pages into a huge page, it must pick one node - which may create cross-NUMA memory access for part of the workload.

Check your NUMA topology first:

numactl --hardware
# Note the node distances - anything above 20 means significant cross-NUMA penalty

# Check per-NUMA huge page allocation
cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
cat /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

# For static huge pages on NUMA, allocate per-node
echo 2048 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 2048 | sudo tee /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

# Bind a process to a NUMA node and its local memory to avoid cross-node THP
numactl --cpunodebind=0 --membind=0 ./your_application