Fail2ban Architecture: What Actually Happens on a Ban

Before writing a single custom jail, understand the data flow. Fail2ban's main process (fail2ban-server) spawns a separate thread per jail. Each jail has a backend (systemd journal, polling, or pyinotify) that feeds log lines to a filter. The filter applies compiled regex patterns to extract a failure timestamp and a source IP. When the failure count exceeds maxretry within findtime seconds, the jail calls its configured action.

Actions are shell scripts in /etc/fail2ban/action.d/. The default iptables-multiport action inserts a rule into the fail2ban chain. The ban persists in memory and in /var/lib/fail2ban/fail2ban.sqlite3. On restart, Fail2ban reads that SQLite file and re-inserts active bans. If that file gets corrupted or deleted, you lose ban state across restarts.

To check what backend a running jail actually uses, run:

fail2ban-client get sshd backend

On systems with systemd, the answer is usually 'systemd'. On older installs or containers without journald, it falls back to 'polling'. This matters because systemd backend reads from the journal and does not need logpath, while polling requires an explicit logpath and the log file must exist at jail start time.

fail2ban-client status
fail2ban-client status sshd
fail2ban-client get sshd backend
fail2ban-client get sshd maxretry

jail.local vs jail.d: The Right Way to Organize Overrides

Never edit jail.conf directly. That file gets overwritten on package upgrades. Use /etc/fail2ban/jail.local for global defaults and individual files under /etc/fail2ban/jail.d/ for per-service configuration. Files in jail.d are loaded alphabetically after jail.local, so naming matters.

We use a numeric prefix convention on our test servers: 00-defaults.conf for global settings, 10-sshd.conf, 20-nginx.conf, 30-postfix.conf, and so on. This makes load order explicit and makes it easy to disable a jail by renaming the file without deleting it.

The [DEFAULT] section in jail.local sets inherited values for every jail. These are the settings we actually run in production:

[DEFAULT]
backend = systemd
bantime = 1h
findtime = 10m
maxretry = 5
bantime.increment = true
bantime.factor = 2
bantime.maxtime = 4w
bantime.overalljails = true
ignoreip = 127.0.0.1/8 ::1 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
action = %(action_mwl)s

Exponential Ban Times with bantime.increment

bantime.increment is the most underused feature in Fail2ban. With it enabled and bantime.factor set to 2, a host that gets banned repeatedly faces doubling ban durations: 1h, 2h, 4h, 8h, up to the bantime.maxtime ceiling. Setting bantime.overalljails = true means failures across all jails count toward the increment, not just within a single jail.

To verify the calculated ban time for a specific IP before it triggers a ban, use:

fail2ban-client get sshd bantime

For an IP already in the database with prior offenses, check the SQLite file directly:

sqlite3 /var/lib/fail2ban/fail2ban.sqlite3 'SELECT ip, timeofban, bantime, bancount FROM bans WHERE ip = "203.0.113.45" ORDER BY timeofban DESC LIMIT 5;'

In our experience, bantime.increment alone reduces repeat offenders by roughly 60-70% compared to a flat ban time, because automated scanners typically have retry logic timed against short ban windows.

One important gotcha: if you set bantime.maxtime shorter than bantime, the increment system silently uses bantime as the floor. Always set bantime.maxtime to something significantly larger than your initial bantime.

sqlite3 /var/lib/fail2ban/fail2ban.sqlite3 \
  'SELECT ip, timeofban, bantime, bancount FROM bans \
   WHERE ip = "203.0.113.45" ORDER BY timeofban DESC LIMIT 5;'
// advertisement

Writing Custom Filters: Regex That Actually Works

Custom filters live in /etc/fail2ban/filter.d/ as .conf files. Each filter must define at least one failregex. The special token expands to the IPv4/IPv6 matching group. Fail2ban compiles these as Python re patterns, so standard Python regex syntax applies.

Here is a real filter we wrote for a Go-based API service that logs JSON:

The filter targets log lines like: 2026-03-15T14:22:01Z [ERROR] auth failed for user admin from ip=203.0.113.45

After writing a filter, always test it with fail2ban-regex before loading it into a jail. This command runs the filter against a real log file and reports match rate, capture groups, and any regex errors:

# /etc/fail2ban/filter.d/myapi-auth.conf
[Definition]
failregex = ^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z \[ERROR\] auth failed .* from ip=$
ignoreregex = ^.* from ip=127\.0\.0\.1$

# Test against a log file
fail2ban-regex /var/log/myapi/app.log /etc/fail2ban/filter.d/myapi-auth.conf

# Test against systemd journal for a specific unit
fail2ban-regex 'systemd-journal' /etc/fail2ban/filter.d/myapi-auth.conf \
  --journalmatch '_SYSTEMD_UNIT=myapi.service'

fail2ban-regex Output: Reading the Numbers

The fail2ban-regex output has three numbers that matter: Lines: X matched, Y missed, Z ignored. If matched is zero, your regex is wrong. If missed is very high relative to matched, your regex is too narrow. If ignored is high, check your ignoreregex and ignoreip settings.

The --print-all-matched flag dumps every matched line with the extracted IP and timestamp:

fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/nginx-botsearch.conf --print-all-matched 2>&1 | head -40

A common failure mode is timezone handling. Fail2ban expects log timestamps in a format it can parse. If your log uses Unix epoch integers, add a datepattern to your filter:

[Definition] datepattern = {EPOCH}

For logs with no timestamp at all (some application logs write to stderr without dates), use the backend's timestamp. With systemd backend, Fail2ban uses the journal entry timestamp automatically, which is usually what you want.

fail2ban-regex /var/log/nginx/access.log \
  /etc/fail2ban/filter.d/nginx-botsearch.conf \
  --print-all-matched 2>&1 | head -40

Multi-Service Jails and the logpath Glob

One jail can monitor multiple log files using glob patterns in logpath. This is useful for services that write per-vhost logs:

logpath = /var/log/nginx/*/error.log /var/log/nginx/error.log

Fail2ban evaluates the glob at jail start. If new log files appear after startup (new vhost added), you must reload the jail to pick them up:

fail2ban-client reload nginx-auth

For services where you want separate ban counts per service but a shared IP ban list, use multiple jails that all write to the same action chain. With iptables-multiport, multiple jails can ban the same IP independently, which means the IP gets doubly banned. That is usually harmless but clutters iptables -L. Use the same chain name across jails to merge them, or rely on nftables with a shared set.

On RHEL 9 and Ubuntu 24.04, we recommend switching to the nftables action:

# /etc/fail2ban/jail.d/10-sshd.conf
[sshd]
enabled = true
port = ssh,2222
filter = sshd
backend = systemd
logpath = /var/log/auth.log
maxretry = 3
bantime = 2h
action = nftables-multiport[name=sshd, port="ssh,2222", protocol=tcp]
         sendmail-whois[name=sshd, dest=ops@example.com]
// advertisement

nftables Action: Why to Switch from iptables

The iptables-multiport action creates a separate chain and inserts rules linearly. With thousands of banned IPs, iptables rule traversal becomes a performance issue. On our test server with 5,000 active bans, iptables -L took 4.2 seconds. With nftables using a set, lookup is O(1) via a hash table.

The nftables-multiport action is included in Fail2ban 0.11+ and 1.0+. Check it exists:

ls /etc/fail2ban/action.d/nftables-multiport.conf

Before enabling it, verify nftables is running and the fail2ban table does not conflict with your existing ruleset:

nft list tables

Fail2ban creates its own table called fail2ban when the first jail with an nftables action starts. It drops that table on service stop. If you have an existing nftables setup, add a rule in your main ruleset to jump to the fail2ban chain:

nft add rule inet filter input ip saddr @fail2ban-sshd drop

However, Fail2ban manages this automatically if you use the nftables-allports or nftables-multiport action - it inserts the drop rule into the fail2ban table's chain, not your main filter table.

# Verify nftables set after banning a test IP
fail2ban-client set sshd banip 203.0.113.1
nft list set inet fail2ban fail2ban-sshd
fail2ban-client set sshd unbanip 203.0.113.1
nft list set inet fail2ban fail2ban-sshd

The Recidive Jail: Banning Persistent Offenders Long-Term

The recidive jail reads Fail2ban's own log file and bans IPs that get banned repeatedly across any other jail. It is a second-order jail - it watches the watcher. Enable it like this:

The default filter for recidive looks for lines matching 'Ban ' in /var/log/fail2ban.log. It bans for 1 week by default after 5 bans within 1 day. We tighten those numbers in production:

Recidive requires that fail2ban.log actually exists and is being written. On systemd-only setups where you redirect all logging to the journal, you need to configure logtarget explicitly:

In /etc/fail2ban/fail2ban.local: logtarget = /var/log/fail2ban.log

After that change, restart Fail2ban. Confirm the log is being written:

tail -f /var/log/fail2ban.log

One production gotcha: logrotate can truncate fail2ban.log mid-session. Configure logrotate to send Fail2ban a HUP signal after rotation so it reopens the file handle:

# /etc/fail2ban/jail.d/99-recidive.conf
[recidive]
enabled = true
filter = recidive
logpath = /var/log/fail2ban.log
action = nftables-allports[name=recidive, protocol=all]
         sendmail-whois-lines[name=recidive, dest=ops@example.com, logpath=%(logpath)s]
maxretry = 3
findtime = 12h
bantime = 4w

# /etc/logrotate.d/fail2ban
/var/log/fail2ban.log {
    weekly
    rotate 8
    compress
    postrotate
        fail2ban-client flushlogs 1>/dev/null
    endscript
}

Custom Actions: Webhook Notifications and External IP Blocklists

The action_mwl default sends email with whois output. In 2026 that is often insufficient - teams want Slack alerts, PagerDuty triggers, or pushes to a shared blocklist. Writing a custom action is straightforward.

An action file has sections: [Definition] with actionstart, actionstop, actioncheck, actionban, and actionunban. Each is a shell command. Variables like , , and are substituted at execution time.

Here is a webhook action we use with a monitoring stack. For teams building more complex automated response pipelines, tools like taskbotshub.ai can orchestrate multi-step incident workflows triggered by these ban events - routing alerts through different channels based on threat severity or IP reputation data.

The curl call below posts to a webhook endpoint. Add it to your jail's action list alongside the nftables ban:

# /etc/fail2ban/action.d/webhook-notify.conf
[Definition]
actionban = curl -s -X POST https://hooks.example.com/fail2ban \
  -H 'Content-Type: application/json' \
  -d '{"jail":"","ip":"","failures":"","time":"
// advertisement

Syncing Bans Across Multiple Servers

Fail2ban has no built-in clustering. If you run 10 web servers, a scanner hitting server 1 can freely hit servers 2-10 until it accumulates enough failures on each. There are three practical approaches.

First: a central log aggregator. Ship all auth logs to a central syslog server. Run Fail2ban there with polling backend against the aggregated logs. Push bans via the webhook action to all nodes using a small script that calls fail2ban-client on each server over SSH.

Second: shared nftables sets via a central Redis or etcd. Write a custom action that writes banned IPs to Redis. Each server runs a daemon that reads from Redis and updates a local nftables set. We have used this pattern and it adds about 200ms propagation latency.

Third: use a commercial or open source WAF layer (HAProxy with a shared stick-table, or Nginx with lua-resty-limit-req backed by Redis) in front of all servers. Fail2ban then only handles SSH and non-HTTP services on each node.

For teams managing named infrastructure across many servers, keeping ban list naming consistent matters. Clear naming conventions for your jails, actions, and blocklist identifiers - similar to how nicename.me approaches structured naming for online identities - reduces operational confusion when you are debugging a ban at 2am.

For the SSH sync approach, this script runs from cron on the central server every 5 minutes:

#!/bin/bash
# sync-bans.sh - run on central Fail2ban server
SERVERS=("web01.example.com" "web02.example.com" "web03.example.com")
JAIL="recidive"

BANNED=$(fail2ban-client status "$JAIL" | grep 'Banned IP' | cut -d: -f2 | tr ',' ' ')

for SERVER in "${SERVERS[@]}"; do
  for IP in $BANNED; do
    ssh -o ConnectTimeout=5 "$SERVER" \
      "fail2ban-client set $JAIL banip $IP" 2>/dev/null
  done
done

Tuning Filter Performance on High-Traffic Servers

On a server processing 50,000 log lines per minute, regex performance matters. Python's re module compiles patterns once at jail start, but a complex alternation with many branches can still introduce measurable latency.

Profile your filter by running fail2ban-regex against a large log sample and timing it:

time fail2ban-regex /var/log/nginx/access.log.1 /etc/fail2ban/filter.d/nginx-limit-req.conf > /dev/null

On our test server with a 200MB access log, a well-written filter takes 8-12 seconds. A poorly anchored pattern with catastrophic backtracking potential took 90+ seconds for the same file. Always anchor your regex. Start with ^ and end with $. Avoid .* in the middle of patterns where a more specific character class works.

If you have multiple similar jails watching the same log file, Fail2ban reads that file once per jail. With 5 jails on /var/log/nginx/access.log, the file is read 5 times. Consider combining them into a single jail with multiple failregex entries - though this means they share maxretry and findtime values.

The usedns option controls whether Fail2ban does reverse DNS lookups on banned IPs. Default is warn, which performs lookups only for whois actions. Set usedns = no globally if you do not use mail actions, eliminating DNS overhead entirely:

usedns = no in [DEFAULT] of jail.local.

time fail2ban-regex /var/log/nginx/access.log.1 \
  /etc/fail2ban/filter.d/nginx-limit-req.conf > /dev/null

# Check Fail2ban CPU usage during log scan
pidstat -p $(pgrep -f fail2ban-server) 1 10

Debugging Bans and Testing Jail Logic

When a jail is not banning as expected, the diagnostic sequence is: check if the jail is running, verify the filter matches your log format, confirm the action executes, and check for IP in ignoreip.

fail2ban-client status shows current ban count and active banned IPs. If the jail shows 0 failures despite obvious attack traffic in the log, the filter is not matching.

To test ban/unban manually without waiting for real failures:

fail2ban-client set sshd banip 203.0.113.99 iptables -L f2b-sshd -n # or: nft list set inet fail2ban fail2ban-sshd fail2ban-client set sshd unbanip 203.0.113.99

To increase log verbosity temporarily for a specific jail, set loglevel to DEBUG in fail2ban.local and restart. Fail2ban will log every line it processes, every regex match attempt, and every action call. Warning: this generates significant log volume on busy servers. Revert after debugging.

For persistent issues with bans not applying, check whether the IP falls in ignoreip - this is a common cause of confusion when testing from your own office IP range. The fail2ban-client command does not warn you when a manual banip is ignored due to ignoreip.

# Full debug sequence
fail2ban-client status sshd
fail2ban-client get sshd ignoreip
fail2ban-client set sshd banip 203.0.113.99
nft list set inet fail2ban fail2ban-sshd
fail2ban-client set sshd unbanip 203.0.113.99

# Check if an IP is already banned
fail2ban-client get sshd banned 203.0.113.99
// advertisement

Monitoring Fail2ban with Prometheus and Grafana

fail2ban-exporter (available on GitHub as fail2ban-prometheus-exporter) exposes jail metrics as Prometheus gauges. Install it as a systemd service alongside Fail2ban:

wget https://github.com/jangrewe/prometheus-fail2ban-exporter/releases/download/v0.5.0/fail2ban_exporter_linux_amd64 chmod +x fail2ban_exporter_linux_amd64 mv fail2ban_exporter_linux_amd64 /usr/local/bin/fail2ban-exporter

The exporter communicates with fail2ban-server via the Unix socket at /var/run/fail2ban/fail2ban.sock. It must run as root or a user with access to that socket.

Key metrics to alert on: - fail2ban_banned_ips{jail="sshd"} sudden spikes indicate coordinated scanning - fail2ban_failed_total increases faster than normal baseline - fail2ban_enabled{jail="recidive"} = 0 means recidive jail is down

For DevOps teams running automated security pipelines, integrating these metrics into your incident response workflow - whether through custom scripts or platforms like taskbotshub.ai that can trigger remediation actions based on metric thresholds - closes the loop between detection and response without manual intervention.

A useful Grafana alert: if banned IPs in any jail exceed 100 within 5 minutes, page on-call. That threshold indicates either a real attack or a broken filter matching legitimate traffic.

# systemd unit for fail2ban-exporter
[Unit]
Description=Fail2ban Prometheus Exporter
After=fail2ban.service

[Service]
ExecStart=/usr/local/bin/fail2ban-exporter \
  --web.listen-address=:9191 \
  --fail2ban.socket=/var/run/fail2ban/fail2ban.sock
Restart=on-failure
User=root

[Install]
WantedBy=multi-user.target