How the TLB Bottleneck Creates the Problem

The Translation Lookaside Buffer is a small, fast cache inside the CPU that maps virtual addresses to physical addresses. On a modern x86-64 processor, the L1 TLB typically holds 64 entries for 4KB pages. Each entry covers 4KB of addressable memory, so 64 entries cover 256KB total. A PostgreSQL shared_buffers of 32GB requires roughly 8 million page table entries. Even with a 1536-entry L2 TLB, you will miss on nearly every memory access that touches a new cache line outside a narrow working set.

Huge pages change this equation. A 2MB huge page covers 512 times more memory per TLB entry. With 64 L1 TLB entries covering 2MB pages, you map 128MB before hitting a miss. For workloads with large working sets, this is the difference between the TLB being useful and being irrelevant. Intel's 1GB pages (available since Nehalem) push this further, though 1GB pages have more allocation constraints and are rarely used outside HPC environments.

The kernel's perf tooling can show you whether TLB misses are actually your bottleneck before you change anything:

perf stat -e dTLB-load-misses,dTLB-store-misses,iTLB-load-misses -p $(pgrep postgres | head -1) sleep 10

Static Huge Pages vs Transparent Huge Pages

Linux provides two mechanisms. Static huge pages (also called HugeTLBfs pages) are reserved at boot or runtime from the buddy allocator. They are never swapped, never compacted, and always available when requested via mmap with MAP_HUGETLB or via a hugetlbfs mount. This is what PostgreSQL uses when huge_pages = on is set in postgresql.conf.

Transparent Huge Pages (THP) are managed by khugepaged, a kernel thread that opportunistically collapses standard 4KB pages into 2MB huge pages without application changes. THP sounds like a free lunch but has a documented latency problem: the collapse operation requires scanning and locking page ranges, which introduces unpredictable stalls. For latency-sensitive workloads like Redis or any real-time processing, THP is usually the wrong choice. For batch workloads like Hadoop or Spark, THP often helps.

Check your current THP setting:

cat /sys/kernel/mm/transparent_hugepage/enabled
# Output example: always [madvise] never

cat /sys/kernel/mm/transparent_hugepage/defrag

Allocating Static Huge Pages at Runtime and Boot

Static huge pages are allocated from physically contiguous memory. The longer the system has been running, the more memory fragmentation exists, and the harder allocation becomes. Allocate them early, ideally before your database starts.

Check current allocation status:

grep -i hugepage /proc/meminfo
# HugePages_Total:    1024
# HugePages_Free:     1024
# HugePages_Rsvd:        0
# HugePages_Surp:        0
# Hugepagesize:       2048 kB
# Hugetlb:         2097152 kB
// advertisement

Setting the Huge Page Count

To set huge pages at runtime, write to /proc/sys/vm/nr_hugepages. For a PostgreSQL server with shared_buffers = 16GB, you need at least 8192 huge pages of 2MB each, plus a buffer for overhead. We typically allocate 10% extra:

Calculate pages needed: 16384 MB / 2 MB per page = 8192 pages. Add 10% buffer: 9012 pages.

# Allocate at runtime
echo 9012 > /proc/sys/vm/nr_hugepages

# Verify allocation succeeded
grep HugePages_Total /proc/meminfo

# Make persistent via sysctl
echo 'vm.nr_hugepages = 9012' >> /etc/sysctl.d/99-hugepages.conf
sysctl -p /etc/sysctl.d/99-hugepages.conf

# For NUMA systems, set per-node allocation
echo 4506 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 4506 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

NUMA Considerations for Huge Page Allocation

On NUMA systems, huge pages allocated via /proc/sys/vm/nr_hugepages are distributed across nodes in proportion to free memory. This is usually fine, but if your database process is pinned to a specific NUMA node via numactl, you want the huge pages on the same node. A cross-node memory access (remote NUMA access) adds 40-100ns of latency depending on the interconnect, which can negate some of the TLB benefit.

Check NUMA topology and per-node huge page allocation:

numactl --hardware

# Check per-node allocation
for node in /sys/devices/system/node/node*/hugepages/hugepages-2048kB/; do
  echo "$node: $(cat ${node}nr_hugepages) allocated, $(cat ${node}free_hugepages) free"
done

# Run postgres on node 0 using node 0 memory
numactl --cpunodebind=0 --membind=0 postgres -D /var/lib/postgresql/16/main

Configuring PostgreSQL to Use Huge Pages

PostgreSQL has supported huge pages since version 9.4 via the huge_pages parameter. Set it to on (not try) if you have allocated enough static huge pages. The try setting falls back to regular pages silently, which makes it harder to diagnose misconfigurations in production.

PostgreSQL maps shared_buffers using a single shared memory segment. The kernel will use huge pages for this segment if huge_pages = on and the segment size is a multiple of the huge page size.

# postgresql.conf
shared_buffers = 16GB
huge_pages = on

# After editing, verify it took effect after restart
psql -c "SHOW huge_pages;"
psql -c "SELECT name, setting FROM pg_settings WHERE name = 'huge_pages';"

# Confirm the process is actually using huge pages
grep -i huge /proc/$(pgrep -x postgres | head -1)/smaps_rollup
// advertisement

Redis and Huge Pages: Disable THP, Enable Static Pages Carefully

Redis's performance profile is different from PostgreSQL. Redis uses fork() for RDB persistence and AOF rewriting. When a child process forks, the kernel uses copy-on-write semantics. With THP enabled, a single write to a 2MB huge page during the fork causes the entire 2MB page to be copied, not just the 4KB that was modified. On a write-heavy Redis instance, this creates massive memory amplification during persistence operations.

The Redis documentation explicitly recommends disabling THP. We measured a 3x increase in fork latency and 40% higher memory usage during BGSAVE on a 32GB Redis instance with THP set to always versus never.

For Redis, disable THP system-wide and use a startup script to enforce it:

# Disable THP immediately
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

# Make persistent via rc.local or a systemd unit
# /etc/systemd/system/disable-thp.service
[Unit]
Description=Disable Transparent Huge Pages
DefaultDependencies=no
After=sysinit.target local-fs.target
Before=basic.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'
ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/defrag'

[Install]
WantedBy=basic.target

Java Virtual Machine Huge Page Configuration

The JVM can use huge pages via two mechanisms: -XX:+UseLargePages (which uses static HugeTLBfs pages) and -XX:+UseTransparentHugePages (which relies on THP). For latency-sensitive JVM workloads, use the static approach with -XX:+UseLargePages and pre-allocate pages.

The JVM process needs permission to use huge pages. Either run as root (bad idea in production) or add the JVM user to the group that owns the hugetlbfs mount, or set vm.hugetlb_shm_group in sysctl.

For a 16GB JVM heap with 2MB pages, you need 8192 pages reserved for the heap alone, plus additional pages for metaspace and the code cache.

# Find the GID for your JVM user
id -g jvmuser

# Allow that group to use huge pages
echo 'vm.hugetlb_shm_group = 1001' >> /etc/sysctl.d/99-hugepages.conf
sysctl -p /etc/sysctl.d/99-hugepages.conf

# Mount hugetlbfs
mkdir -p /mnt/hugepages
mount -t hugetlbfs -o gid=1001,mode=1770 hugetlbfs /mnt/hugepages

# JVM flags
java -Xms16g -Xmx16g \
     -XX:+UseLargePages \
     -XX:LargePageSizeInBytes=2m \
     -XX:+AlwaysPreTouch \
     -jar application.jar

# Verify huge page use after JVM start
grep AnonHugePages /proc/$(pgrep -f application.jar)/smaps_rollup

1GB Huge Pages for HPC and Memory-Bound Workloads

On Intel and AMD processors, 1GB pages (gigantic pages) are available if the CPU supports the pdpe1gb flag. These are configured separately from 2MB pages and cannot be allocated after boot since they require physically contiguous 1GB regions that are only available during early boot before memory fragmentation occurs.

To allocate 1GB pages, pass hugepagesz=1G hugepages=N to the kernel at boot. On GRUB systems:

# Check CPU support
grep pdpe1gb /proc/cpuinfo | head -1

# /etc/default/grub - add to GRUB_CMDLINE_LINUX
GRUB_CMDLINE_LINUX="hugepagesz=1G hugepages=64 hugepagesz=2M hugepages=512"

# Update GRUB
update-grub   # Debian/Ubuntu
grub2-mkconfig -o /boot/grub2/grub.cfg  # RHEL/Fedora

# After reboot, verify
grep -i 'Hugepagesize\|HugePages_Total' /proc/meminfo
ls /sys/kernel/mm/hugepages/
// advertisement

Monitoring Huge Page Usage and Fragmentation

Knowing you have huge pages configured is not the same as knowing they are being used. Check /proc/meminfo for HugePages_Rsvd (reserved but not yet faulted in) versus HugePages_Free (available). A high Rsvd count with low usage means applications have reserved pages but not touched them yet, which is normal with lazy allocation.

For per-process huge page accounting, smaps provides the detail:

# System-level overview
watch -n 2 'grep -E "HugePages|Hugetlb" /proc/meminfo'

# Per-process huge page use
awk '/^AnonHugePages:/ {sum += $2} END {print sum/1024 " MB of AnonHugePages"}' \
  /proc/$(pgrep postgres | head -1)/smaps

# List all processes using huge pages
for pid in /proc/[0-9]*/smaps; do
  p=$(dirname $pid | xargs basename)
  hp=$(awk '/AnonHugePages/ {s+=$2} END {print s}' $pid 2>/dev/null)
  [ "${hp:-0}" -gt 0 ] && echo "$p $(cat /proc/$p/comm 2>/dev/null): ${hp}kB"
done

# Check compaction statistics (relevant for THP workloads)
grep compact /proc/vmstat

Automating Huge Page Tuning Across Fleets

Manually tuning huge pages on individual servers does not scale. If you manage more than a handful of database or application servers, you need this in your configuration management layer. Ansible roles work well here: calculate the required huge page count based on Ansible facts (ansible_memtotal_mb), write the sysctl configuration, and mount hugetlbfs with the correct permissions.

For teams integrating infrastructure automation into CI/CD pipelines, platforms like taskbotshub.ai can orchestrate these tuning tasks as part of deployment workflows, ensuring huge page configuration is validated before a new database instance receives traffic.

A minimal Ansible task that calculates and applies huge pages based on a fraction of total RAM:

# tasks/hugepages.yml
- name: Calculate huge page count (50% of RAM for DB node)
  set_fact:
    hugepage_count: "{{ ((ansible_memtotal_mb * 0.5) / 2) | int }}"

- name: Set vm.nr_hugepages
  sysctl:
    name: vm.nr_hugepages
    value: "{{ hugepage_count }}"
    state: present
    reload: yes
    sysctl_file: /etc/sysctl.d/99-hugepages.conf

- name: Disable THP for Redis nodes
  shell: echo never > /sys/kernel/mm/transparent_hugepage/enabled
  when: node_role == 'redis'

- name: Ensure THP disable service is installed
  template:
    src: disable-thp.service.j2
    dest: /etc/systemd/system/disable-thp.service
  notify: reload systemd

Benchmarking the Impact: What to Expect

Numbers from our test environment on a dual-socket Xeon Gold 6338 with 512GB RAM, running PostgreSQL 16 with a 128GB shared_buffers on a 400GB TPC-H dataset:

Without huge pages: 42,000 transactions per second under pgbench -c 128 -j 16, with perf showing 18% of CPU cycles lost to TLB misses.

With 65,536 static huge pages (128GB / 2MB): 58,000 transactions per second, TLB miss cycles down to 3%. That is a 38% throughput improvement from a configuration change that takes under five minutes.

The gains are not universal. For workloads with a small working set that fits in the L2 TLB regardless of page size, huge pages add negligible benefit. For workloads doing random I/O against storage, the bottleneck is elsewhere. Profile first with perf stat before committing to a tuning strategy.

Redis benchmark on a 64GB instance, BGSAVE under write load: - THP=always: fork latency 480ms average, memory usage during save 1.6x dataset size - THP=never + no static huge pages: fork latency 28ms average, memory usage 1.05x dataset size

The Redis result is the clearest: THP makes Redis worse in almost every measurable way under write pressure.

# Quick pgbench baseline before and after huge page change
pgbench -i -s 500 benchdb
pgbench -c 128 -j 16 -T 60 benchdb

# Capture TLB stats during the run
perf stat -e dTLB-load-misses,dTLB-store-misses \
  -p $(pgrep -x postgres | head -1) \
  sleep 60 2>&1 | grep -E 'TLB|seconds'
// advertisement

Common Failure Modes and How to Diagnose Them

The most common failure is PostgreSQL logging 'WARNING: could not use huge pages, reducing request' and silently falling back to standard pages. This happens when huge_pages = try (not on), when the allocated page count is insufficient, or when the kernel.shmmax sysctl is too low for the shared memory segment.

Check that shared memory limits are not blocking huge page allocation:

# shared memory must accommodate the full shared_buffers allocation
sysctl kernel.shmmax
sysctl kernel.shmall

# For 16GB shared_buffers, shmmax must be >= 17179869184
echo 'kernel.shmmax = 17179869184' >> /etc/sysctl.d/99-hugepages.conf
echo 'kernel.shmall = 4294967296' >> /etc/sysctl.d/99-hugepages.conf
sysctl -p /etc/sysctl.d/99-hugepages.conf

# Check dmesg for huge page allocation failures
dmesg | grep -i huge

# Check if allocation partially failed (Total < requested)
grep HugePages_Total /proc/meminfo

# Force memory compaction to aid allocation on a running system
echo 1 > /proc/sys/vm/compact_memory