Flask on Lessons learned

Dynamic DNS for home routers the hacker way

Thu, 28 Oct 2021 17:48:15 +0200

Skip public DDNS services: Python can handle the same job. This post shows how to modify a DNS record in a zone file, upload the zone over SSH, and expose the update as a small web service.

The setup

A MikroTik router with a dynamic public IPv4 address
A name for the router in my own zone, on my primary DNS server
Python with dnspython, paramiko, Flask, and flask_jwt_extended

I first looked at dynamic DNS over a standard protocol, but I had to abandon the idea. MikroTik also offers its cloud service for a similar result, but I find it less convenient than a name in my own zone. That led me to a way to modify the zone file with Python.

The examples cover IPv4 only. Extending them to IPv6 is a small change.

Modify the zone file

The update has three steps: open the zone file, modify the record, and save the file.

def update(zone, name, newip4):
 original = f'zones/{zone}.zone'
 updated = update_dynamic_ip4(original, name, newip4, 3600)
 with open(original, 'w') as file:
 file.write(f'$ORIGIN {zone}.\n')
 file.write(updated)

The record update itself:

Load the zone file.
Find the record.
Replace it, or stop if the address hasn’t changed.
Update the SOA serial.
Return the zone as text.

Don’t forget the SOA serial. Without the bump, the secondary servers of your zone don’t see the change.

dnspython does the job, but its API is not my favorite cup of tea:

from datetime import date

import dns.rdataclass
import dns.rdataset
import dns.rdatatype
import dns.zone


class NoUpdate(ValueError):
 pass


def update_dynamic_ip4(zonefile, name, newip4, ttl):
 z = dns.zone.from_file(zonefile)
 new_data = dns.rdataset.from_text(dns.rdataclass.IN, dns.rdatatype.A, ttl, newip4)
 node = z.find_node(name)
 oldip4 = node.get_rdataset(dns.rdataclass.IN, dns.rdatatype.A).to_text().split()[3]
 if str(oldip4) == newip4:
 raise NoUpdate
 node.replace_rdataset(new_data)

 soa = z.find_node("@").find_rdataset(dns.rdataclass.IN, dns.rdatatype.SOA)
 soa_updated = update_soa(soa)
 z.find_node("@").replace_rdataset(soa_updated)
 return z.to_text(relativize=False)

I keep the serial as a date (%Y%m%d) and a two-digit revision. If the serial is from today, the revision increases by one; otherwise, it starts at zero for today. That allows 100 changes a day, plenty for one dynamic address.

def update_soa(soa):
 soa_parts = list(soa.to_text().split())
 version = soa_parts[-5]
 today = date.today().strftime("%Y%m%d")
 increment = 0
 if version.startswith(today):
 increment = int(version[len(today):]) + 1
 version = f'{today}{increment:02d}'
 soa_parts[-5] = version
 soa_new = " ".join(soa_parts[3:])
 ttl = soa_parts[0]
 return dns.rdataset.from_text(dns.rdataclass.IN, dns.rdatatype.SOA, ttl, soa_new)

Upload the zone file and reload the DNS server

By hand, you’d SSH to the DNS server, replace the file, and reload the server. paramiko does the same:

import paramiko


def upload_zone(localfile, remotefile, remotehost, remoteport, keyfile):
 with paramiko.SSHClient() as client:
 client.load_system_host_keys()
 client.connect(remotehost, remoteport, key_filename=keyfile)
 with client.open_sftp() as ftp:
 ftp.put(localfile, remotefile)
 client.exec_command('nsd-reload') # the reload command of your DNS server

The DNS server’s host key must already be in your known_hosts.

Expose it as a REST API

The router sends a request, and the app updates the zone. The new address comes from the request itself, and the router identifies itself with a JWT. remotefile, remotehost, remoteport, and keyfile come from the configuration.

This example creates a new secret and token each time the app starts, so the router needs the new token after a restart:

import uuid

from flask import Flask, request
from flask_jwt_extended import JWTManager, create_access_token, get_jwt_identity, jwt_required

app = Flask(__name__)
app.config["JWT_SECRET_KEY"] = str(uuid.uuid4())
jwt = JWTManager(app)
with app.app_context():
 token = create_access_token(identity={'name': name, 'zone': zone},
 expires_delta=False)

The endpoint:

@app.route('/api/update/ipv4', methods=['POST'])
@jwt_required()
def update_endpoint():
 identity = get_jwt_identity()
 name = identity['name']
 zone = identity['zone']
 if 'X-Forwarded-For' in request.headers:
 newip = request.headers['X-Forwarded-For']
 else:
 newip = request.remote_addr
 try:
 update(zone, name, newip)
 except NoUpdate:
 return {'status': 'unchanged'}
 upload_zone(f'zones/{zone}.zone', remotefile, remotehost, remoteport, keyfile)
 return {'status': 'updated'}

Most periodic calls find the address unchanged, so NoUpdate is the common path, not an error. Read X-Forwarded-For only when the app runs behind your own reverse proxy that sets it; otherwise, any client can put an arbitrary address there. Before deploying, add input validation and handling for the remaining exceptions.

Once deployed, set up the router to call the endpoint periodically.

Flask @cached decorator traps: POST gets the cached GET, no-store is stored, and X-Last-Modified: None

Fri, 19 Oct 2018 21:26:00 +0200

Update 2026-11-08: I rebuilt every repro on Flask 2.2.3 with Werkzeug 2.2.3 and Flask-Caching 2.0.2, and on Flask 3.1.3 with Werkzeug 3.1.9 and Flask-Caching 2.5.1, under Python 3.11.

A Flask view decorator I kept for years failed in a way that only showed up when a POST got the body of an earlier GET. The view never ran, and the client got a 200:

GET /item 200 "GET ran" X-Cache: MISS
POST /item 200 "GET ran" X-Cache: HIT from Server

The cache key was request.full_path, so any two requests with the same URL shared an entry, regardless of method, user, or Accept header. To fix this, let only anonymous GET and HEAD requests touch the shared entry, build the key from the response inputs, and leave the HTTP headers to Werkzeug.

The decorator entered a 2018 project with a docstring pointing at a Code Review question and Jake Archibald’s caching article . I rebuilt each case from rereading it. My routes were all GET-only and public, so the POST and private cases never fired, and I have no logs showing what the others cost anyone.

A decorator with one shared entry per representation

from functools import wraps
from urllib.parse import urlencode

from flask import make_response, request
from flask_caching import Cache

cache = Cache()


def cached(params=(), offers=("application/json",), server_timeout=300):
 """Server-cache a public view. Declare the query parameters it reads."""

 def decorator(view):
 @wraps(view)
 def wrapper(*args, **kwargs):
 if (
 request.method not in ("GET", "HEAD")
 or request.authorization
 or request.cookies
 ):
 return view(*args, **kwargs)

 query = urlencode([(p, request.args.getlist(p)) for p in params], doseq=True)
 media_type = request.accept_mimetypes.best_match(offers, offers[0])
 key = f"view/{request.path}?{query}#{media_type}"

 response = cache.get(key)
 if response is None:
 response = make_response(view(*args, **kwargs))
 if response.status_code != 200:
 return response
 response.add_etag()
 response.cache_control.public = True
 response.cache_control.no_cache = True
 response.vary.add("Accept")
 cache.set(key, response, timeout=server_timeout)
 return response.make_conditional(request)

 return wrapper

 return decorator

Both decorators show the same probes, identical on both version sets:

Probe	Old decorator	Fixed decorator
POST after GET	Cached GET body, view skipped	View runs
Same URL, cookies for two users	Second user sees the first user’s page	Each sees their own
Anonymous request after a logged-in one	200 with the logged-in body	401
JSON, then `Accept: text/turtle`	JSON for both	Turtle for the second
100 requests with `Cache-Control: no-cache`	100 view runs	0 after the first
100 requests with a junk `?x=N`	100 view runs, 100 entries	0 after the first, 1 entry
`If-None-Match` with the `W/` form of the tag	200	304
304 headers	No `ETag`, `X-Last-Modified: None`	`ETag`, `Cache-Control`, `Vary`

Two caveats. Bypassing any cookie is blunt, since one analytics cookie turns the cache off for that browser, so narrow it to your session cookie. And params must list every argument the view reads, or two different URLs collide again. For per-user caching, put the user ID in the key and send private.

`no-store` and `private` don’t bind your own cache

The bare @cached() sent Cache-Control: no-store and still called cache.set:

GET /nostore 200 Cache-Control: no-store X-Cache: MISS
GET /nostore 200 Cache-Control: no-store X-Cache: HIT from Server

no-store and private are instructions to the browser and to proxies. An entry your application writes to Redis is outside their reach. The arguments cacheable=False and client_only=True read like server switches and only picked a header. With private, the second user received hello alice from the shared entry.

Decorator order makes it worse. With @cached above a login check, a hit never reaches the check: after one logged-in request, an anonymous client got a 200 with account of alice.

The method guard covered writes only

request.method in ["GET", "HEAD"] protected cache.set. The cache.get above it had no guard. A POST on a cold cache runs the view, so a test that only sends POST passes.

Flask-Caching’s @cache.cached(timeout=60) did the same on a GET and POST route, in both versions. Add unless=lambda: request.method != "GET" there.

Any client could pick the miss

if response is not None and request.headers.get("Cache-Control", "") != "no-cache":

The comment says “respect the hard-refresh”. It also hands every client a way to run the expensive view on demand: 100 requests, 100 runs. The comparison is exact, so 100 requests with no-cache, no-store or max-age=0 ran the view once. A junk query parameter does the same and adds an entry per request.

Your application’s cache is part of the origin, and no RFC asks it to obey a client’s request header. Invalidate from the server with cache.delete. The price: a hard refresh shows the cached body until server_timeout passes.

One typo, one bytes object

The 304 branch read a header that was never written:

response.headers.add("X-Last-Modified", str(now))
headers["X-Last-Modified"] = response.headers.get("X-LastModified") # None

Werkzeug turns None into a string, so every 304 carried X-Last-Modified: None and no error. It also lacked the ETag that RFC 9110 requires on a 304.

The tag was binascii.hexlify(os.urandom(4)), which returns bytes:

Werkzeug	`ETag` value
2.2.3 to 2.3.8	`a79d7c42`
3.0.0 and 3.1.9	`b'77155122'`

Neither has the double quotes that the grammar requires. Both round-trip through If-None-Match unchanged, so the change in Werkzeug 3.0 raises no error. The comparison was a literal ==, where If-None-Match calls for weak comparison , and a random tag changes every time the entry refills even if the body doesn’t. add_etag() hashes the body and make_conditional() does the comparison.

`no-cache` means revalidate

The old comments had this right: “no-cache doesn’t mean ‘don’t cache’” and “must-revalidate doesn’t mean ‘must revalidate’”.

Directive	Stored?	Reused without asking the server?
`no-store`	No	No
`no-cache`	Yes	No
`must-revalidate, max-age=900`	Yes	For 900 s, then it must revalidate

Then every call site passed must_revalidate=True, client_timeout=900, which sent no-cache, public, max-age=900. An unqualified no-cache forces validation on every reuse, so the 900 seconds bought nothing. I had the explanation in the file and still passed both.

Takeaway

A server-side entry goes to everyone whose request maps to its key, so list what the response depends on and put each item in the key or refuse to cache.

Try it on your own cached views: GET a URL, POST to it, then GET it with another user’s cookie, and fail the test if any two bodies match.