<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Fail2ban on Lessons learned</title><link>https://eghuro.cz/tags/fail2ban/</link><description>Recent content in Fail2ban on Lessons learned</description><generator>Hugo</generator><language>en-us</language><copyright>© Alexandr Mansurov · Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)</copyright><lastBuildDate>Fri, 16 Oct 2026 15:33:00 +0200</lastBuildDate><atom:link href="https://eghuro.cz/tags/fail2ban/index.xml" rel="self" type="application/rss+xml"/><item><title>fail2ban that never banned anyone: '/bin/sh: 1: nft: not found'</title><link>https://eghuro.cz/posts/fail2ban-nft-not-found/</link><pubDate>Fri, 16 Oct 2026 15:33:00 +0200</pubDate><guid>https://eghuro.cz/posts/fail2ban-nft-not-found/</guid><description>&lt;p&gt;fail2ban said the address was banned:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;$ sudo fail2ban-client status nginx-limit-req&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;...&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; `- Banned IP list:&#9;203.0.113.10&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;From that address, right after:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;$ curl -s -o /dev/null -w &amp;#39;%{http_code}\n&amp;#39; https://example.com/&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;302&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The reason was in &lt;code&gt;/var/log/fail2ban.log&lt;/code&gt; all along:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;2026-03-14 10:21:05,205 fail2ban.utils [1293]: ERROR 7f5e8fbb0d30 -- exec: nft add table inet f2b-table&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;2026-03-14 10:21:05,206 fail2ban.utils [1293]: ERROR 7f5e8fbb0d30 -- stderr: &amp;#39;/bin/sh: 1: nft: not found&amp;#39;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;On Debian, fail2ban bans through the &lt;code&gt;nft&lt;/code&gt; command. A host that runs firewalld can easily lack it. fail2ban then counts failures, lists bans, and blocks nothing, including the sshd jail.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-setup"&gt;The setup &lt;/h2&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;Debian 13, fail2ban 1.1.0-8, firewalld 2.3.1 with &lt;code&gt;FirewallBackend=nftables&lt;/code&gt;&lt;/li&gt;&#10;&lt;li&gt;nginx as a reverse proxy in front of a few web apps, SSH forwarded from the router&lt;/li&gt;&#10;&lt;li&gt;fail2ban set up by hand, with the stock sshd jail&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&lt;p&gt;Debian&amp;rsquo;s &lt;code&gt;/etc/fail2ban/jail.d/defaults-debian.conf&lt;/code&gt; picks the ban action:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-ini" data-lang="ini"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;[DEFAULT]&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="na"&gt;banaction&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;nftables&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="na"&gt;banaction_allports&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;nftables[type=allports]&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&#10;&lt;h2 id="how-it-showed-up"&gt;How it showed up &lt;/h2&gt;&#10;&lt;p&gt;I was adding jails for the web logins, so I tested them from a VPS outside every whitelisted network. Testing from home proves nothing when home is in &lt;code&gt;ignoreip&lt;/code&gt;. Twenty-five quick POSTs to a login endpoint:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; i in &lt;span class="k"&gt;$(&lt;/span&gt;seq 25&lt;span class="k"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;do&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; curl -s -o /dev/null -w &lt;span class="s1"&gt;&amp;#39;%{http_code} &amp;#39;&lt;/span&gt; -d &lt;span class="nv"&gt;client_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;probe https://example.com/login&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;done&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The first 11 got 400 from the app, the rest 429 from nginx&amp;rsquo;s rate limit, and the &lt;code&gt;nginx-limit-req&lt;/code&gt; jail banned the VPS as it should. Then the next request got a 302, as if nothing had happened.&lt;/p&gt;&#10;&lt;p&gt;&lt;code&gt;sudo nft list table inet f2b-table&lt;/code&gt; answered &lt;code&gt;sudo: nft: command not found&lt;/code&gt;. The log held ten lines like the ones above. Each failed ban left two ERROR lines in a log nobody reads, while &lt;code&gt;fail2ban-client status&lt;/code&gt; still listed the bans.&lt;/p&gt;&#10;&#10;&lt;h2 id="the-fix"&gt;The fix &lt;/h2&gt;&#10;&lt;p&gt;Install the &lt;code&gt;nft&lt;/code&gt; command:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;sudo apt install nftables&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;sudo systemctl restart fail2ban&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Leave &lt;code&gt;nftables.service&lt;/code&gt; disabled, as the package installs it. Debian&amp;rsquo;s &lt;code&gt;/etc/nftables.conf&lt;/code&gt; starts with &lt;code&gt;flush ruleset&lt;/code&gt;, so enabling the service would wipe firewalld&amp;rsquo;s rules at the next boot. fail2ban doesn&amp;rsquo;t need the service, only the binary. It keeps its bans in a table of its own, &lt;code&gt;inet f2b-table&lt;/code&gt;, next to firewalld&amp;rsquo;s &lt;code&gt;inet firewalld&lt;/code&gt;.&lt;/p&gt;&#10;&lt;p&gt;On restart, fail2ban restores the bans it has on record, so the ones that had silently failed now take effect:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;fail2ban.actions [1649343]: NOTICE [nginx-limit-req] Restore Ban 203.0.113.10&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Give it a few seconds before you check. Right after the restart, &lt;code&gt;nft list table inet f2b-table&lt;/code&gt; still said &lt;code&gt;No such file or directory&lt;/code&gt;. A moment later, it showed the set and the rule:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;table inet f2b-table {&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;set addr-set-nginx-limit-req {&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;&#9;type ipv4_addr&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;&#9;elements = { 203.0.113.10 }&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;}&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;chain f2b-chain {&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;&#9;type filter hook input priority filter - 1; policy accept;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;&#9;tcp dport { 80, 443 } ip saddr @addr-set-nginx-limit-req reject with icmp port-unreachable&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&#9;}&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;}&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;This time, curl on the VPS couldn&amp;rsquo;t connect at all (exit code 7). If you manage the host with Ansible or similar, add &lt;code&gt;nftables&lt;/code&gt; to the package list alongside &lt;code&gt;fail2ban&lt;/code&gt; so a rebuilt host doesn&amp;rsquo;t lose it again.&lt;/p&gt;&#10;&#10;&lt;h2 id="why-the-binary-was-missing"&gt;Why the binary was missing &lt;/h2&gt;&#10;&lt;p&gt;firewalld talks to the kernel through the &lt;code&gt;python3-nftables&lt;/code&gt; bindings, not the &lt;code&gt;nft&lt;/code&gt; command, so it doesn&amp;rsquo;t pull in the &lt;code&gt;nftables&lt;/code&gt; package. fail2ban only recommends it as an alternative:&lt;/p&gt;&#10;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;$ apt-cache show fail2ban | grep ^Recommends&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;Recommends: nftables | iptables, whois, python3-pyinotify, python3-setuptools&#10;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;&lt;code&gt;iptables&lt;/code&gt; was already installed (firewalld recommends it), so apt considered the recommendation met. The ban action still called &lt;code&gt;nft&lt;/code&gt;.&lt;/p&gt;&#10;&lt;p&gt;The same thing can happen with any ban action whose command isn&amp;rsquo;t on the host: the action fails, and the address stays on the ban list anyway.&lt;/p&gt;&#10;&#10;&lt;h2 id="takeaway"&gt;Takeaway &lt;/h2&gt;&#10;&lt;p&gt;&lt;code&gt;Banned IP list&lt;/code&gt; shows what fail2ban decided, not what the firewall does. The only proof that a ban works is a blocked connection from the banned address, and &lt;code&gt;grep ERROR /var/log/fail2ban.log&lt;/code&gt; tells you why one isn&amp;rsquo;t.&lt;/p&gt;&#10;&lt;p&gt;Get one of your own hosts to ban a machine outside its whitelist, then try to connect from that machine.&lt;/p&gt;&#10;</description></item></channel></rss>