Firewalld on Lessons learned

fail2ban that never banned anyone: '/bin/sh: 1: nft: not found'

Fri, 16 Oct 2026 15:33:00 +0200

fail2ban said the address was banned:

$ sudo fail2ban-client status nginx-limit-req
...
 `- Banned IP list:	203.0.113.10

From that address, right after:

$ curl -s -o /dev/null -w '%{http_code}\n' https://example.com/
302

The reason was in /var/log/fail2ban.log all along:

2026-03-14 10:21:05,205 fail2ban.utils [1293]: ERROR 7f5e8fbb0d30 -- exec: nft add table inet f2b-table
2026-03-14 10:21:05,206 fail2ban.utils [1293]: ERROR 7f5e8fbb0d30 -- stderr: '/bin/sh: 1: nft: not found'

On Debian, fail2ban bans through the nft command. A host that runs firewalld can easily lack it. fail2ban then counts failures, lists bans, and blocks nothing, including the sshd jail.

The setup

Debian 13, fail2ban 1.1.0-8, firewalld 2.3.1 with FirewallBackend=nftables
nginx as a reverse proxy in front of a few web apps, SSH forwarded from the router
fail2ban set up by hand, with the stock sshd jail

Debian’s /etc/fail2ban/jail.d/defaults-debian.conf picks the ban action:

[DEFAULT]
banaction = nftables
banaction_allports = nftables[type=allports]

How it showed up

I was adding jails for the web logins, so I tested them from a VPS outside every whitelisted network. Testing from home proves nothing when home is in ignoreip. Twenty-five quick POSTs to a login endpoint:

for i in $(seq 25); do
 curl -s -o /dev/null -w '%{http_code} ' -d client_id=probe https://example.com/login
done

The first 11 got 400 from the app, the rest 429 from nginx’s rate limit, and the nginx-limit-req jail banned the VPS as it should. Then the next request got a 302, as if nothing had happened.

sudo nft list table inet f2b-table answered sudo: nft: command not found. The log held ten lines like the ones above. Each failed ban left two ERROR lines in a log nobody reads, while fail2ban-client status still listed the bans.

The fix

Install the nft command:

sudo apt install nftables
sudo systemctl restart fail2ban

Leave nftables.service disabled, as the package installs it. Debian’s /etc/nftables.conf starts with flush ruleset, so enabling the service would wipe firewalld’s rules at the next boot. fail2ban doesn’t need the service, only the binary. It keeps its bans in a table of its own, inet f2b-table, next to firewalld’s inet firewalld.

On restart, fail2ban restores the bans it has on record, so the ones that had silently failed now take effect:

fail2ban.actions [1649343]: NOTICE [nginx-limit-req] Restore Ban 203.0.113.10

Give it a few seconds before you check. Right after the restart, nft list table inet f2b-table still said No such file or directory. A moment later, it showed the set and the rule:

table inet f2b-table {
	set addr-set-nginx-limit-req {
		type ipv4_addr
		elements = { 203.0.113.10 }
	}
	chain f2b-chain {
		type filter hook input priority filter - 1; policy accept;
		tcp dport { 80, 443 } ip saddr @addr-set-nginx-limit-req reject with icmp port-unreachable
	}
}

This time, curl on the VPS couldn’t connect at all (exit code 7). If you manage the host with Ansible or similar, add nftables to the package list alongside fail2ban so a rebuilt host doesn’t lose it again.

Why the binary was missing

firewalld talks to the kernel through the python3-nftables bindings, not the nft command, so it doesn’t pull in the nftables package. fail2ban only recommends it as an alternative:

$ apt-cache show fail2ban | grep ^Recommends
Recommends: nftables | iptables, whois, python3-pyinotify, python3-setuptools

iptables was already installed (firewalld recommends it), so apt considered the recommendation met. The ban action still called nft.

The same thing can happen with any ban action whose command isn’t on the host: the action fails, and the address stays on the ban list anyway.

Takeaway

Banned IP list shows what fail2ban decided, not what the firewall does. The only proof that a ban works is a blocked connection from the banned address, and grep ERROR /var/log/fail2ban.log tells you why one isn’t.

Get one of your own hosts to ban a machine outside its whitelist, then try to connect from that machine.

No DHCP for an LXC container while iptables counts the packets as ACCEPT: firewalld rejects them in its own nftables table

Tue, 13 Oct 2026 22:56:00 +0200

A new LXC container came up with an IPv6 address and no IPv4 address. According to iptables, the host allowed DHCP from the container bridge, and the counter on that rule kept growing:

$ sudo iptables -L INPUT -v -n | grep dpt:67
 0 0 ACCEPT tcp -- lxcbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67
 26 7956 ACCEPT udp -- lxcbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67

Twenty-six packets accepted, and dnsmasq, listening on that bridge, never logged a single DHCPDISCOVER.

With nftables, accept only ends the chain it sits in. Any other base chain on the same hook still gets the packet. On this host, the next one belonged to firewalld, and firewalld rejected everything arriving on an interface that wasn’t in any of its zones.

The setup

Debian 13 host with the stock lxc-net bridge: lxcbr0, 10.0.3.0/24, dnsmasq at 10.0.3.1
firewalld 2.3.1 with FirewallBackend=nftables, default zone public
iptables 1.8.11, the nf_tables flavor Debian installs as iptables
an unprivileged Debian 13 container running systemd-networkd with DHCP=true

When lxc-net brings up the bridge, it adds its own iptables rules: ACCEPT for DNS and DHCP on lxcbr0, ACCEPT for forwarding, and a MASQUERADE for the subnet. It knows nothing about firewalld.

Following the packet

The first visible symptom was the container’s bootstrap script, which spent nine minutes waiting on DNS lookups that timed out. From there, each check moved the problem one step along the path:

The container sent requests. With debug logging, systemd-networkd showed DHCPv4 client: DISCOVER in the selecting state.
The host received them. The ACCEPT counter went from 20 to 26 to 29 while I made the client retry with networkctl reconfigure eth0.
dnsmasq had the right socket open: 0.0.0.0%lxcbr0:67.
dnsmasq received nothing. strace showed it sitting in poll(). During a retry, it read only an IPv6 router solicitation, which it answered. That is where the IPv6 address came from.

AppArmor (no denials), reverse-path filtering (loose on the bridge), br_netfilter (not loaded), leftover iptables-legacy tables (empty), and socket buffer drops (RcvbufErrors 0) all came up clean. The host had no tcpdump, and no nft either:

$ sudo nft list ruleset
sudo: nft: command not found

That left iptables as the only view of the firewall, and iptables shows only the tables it created. So the packet passed an ACCEPT at the input hook and never reached the socket. Listing the running services found a second firewall:

firewalld.service loaded active running firewalld - dynamic firewall daemon

$ sudo firewall-cmd --get-zone-of-interface=lxcbr0
no zone

Two tables on one hook

After apt install nftables (the binary is enough; leave nftables.service disabled), you can see both chains on the input hook. Trimmed:

$ sudo nft list chains
table ip filter {
	chain INPUT {
		type filter hook input priority filter; policy accept;
	}
}
table inet firewalld {
	chain filter_INPUT {
		type filter hook input priority filter + 10; policy accept;
	}
}

ip filter holds lxc-net’s rules, written through iptables-nft. inet firewalld runs right after it, and an interface that no zone claims falls through to the default zone:

chain filter_INPUT_POLICIES {
	...
	jump filter_IN_policy_allow-host-ipv6
	jump filter_IN_public
	reject with icmpx admin-prohibited
}

public allowed SSH and the DHCPv6 client, but no DHCP server and no DNS, so the DISCOVER hit the final reject. The zone chain also ends with meta l4proto { icmp, ipv6-icmp } accept, which is why the router solicitation got through.

The fix

Put the bridge in a firewalld zone. trusted accepts everything, including forwarded traffic, which the containers need for internet access:

sudo firewall-cmd --zone=trusted --add-interface=lxcbr0
sudo firewall-cmd --permanent --zone=trusted --add-interface=lxcbr0

Or with Ansible:

- name: Trust the container bridge in firewalld
 ansible.posix.firewalld:
 zone: trusted
 interface: lxcbr0
 permanent: true
 immediate: true
 state: enabled

The container got its lease on the client’s next retry, without restarting anything. trusted also lets containers reach every port on the host. If that’s too much, give the bridge a zone of its own that allows only DHCP, DNS, and forwarding. The same applies to any interface that another tool creates, such as a VPN tunnel.

Why the counter lied

The nftables wiki says that “an accept verdict … isn’t necessarily final”, while “drops take immediate effect, with no further rules or chains being evaluated.” Base chains on one hook run in priority order. An accept ends one chain, and a drop or reject in any chain wins. When two firewalls share a host, the stricter one decides, and the permissive one keeps counting packets that never arrive.

Most of the time you won’t see the second table. iptables -L lists only what iptables-nft created, and a host running firewalld may not have the nft command at all. (That missing binary also leaves fail2ban unable to ban anyone .)

Takeaway

A counter on an ACCEPT rule only proves that the packet reached that rule. When an accepted packet still doesn’t arrive, list every base chain on the hook and find the later one that drops it.

If your host runs firewalld alongside anything that writes its own iptables rules, run sudo nft list chains | grep hook and count the chains on input and forward.