DNSMasq + Network Manager + Docker = Fun?

So the idea is to try and use my existing tech (out of the box Fedora 33) to take requests for local services and poke them towards docker containers.

In the case of working on work stuff, this is poking requests to a traefik instance in docker, and in the case of my own chicken scratchings, directly to a Caddy server instance.

And after many days and weeks of tearing my hair out, I've got it working as I'd like. Previously it was kinda working for Ubuntu, but Fedora made things a bit more interesting.

Steps

So now follows a number of steps (lets be honest, this is a note for myself so I can reproduce it later):

1. Disable systemd-resolved because, no.
2. Piggyback onto NetworkManager's own instance of DNSMasq.
3. Write a sexy config for NM's DNSMasq to do all the fancy redirects.
4. Release the Niffler!

Step 1

Disable systemd-resolved (hate this thing, and really have no idea what it does other than hog port 53 to itself. But the internet seems to agree it's ok to disable it).

$ sudo systemctl stop systemd-resolved
$ sudo systemctl disable systemd-resolved

Step 2

Turns out NetworkManager has it's own sneaky DNSMasq that will kick off if you enable it. So lets create /etc/NetworkManager/conf.d/00-dnsmasq.conf and dump in:

[main]
dns=dnsmasq

Next time Network Manager starts, a sneaky DNSMasq service will spawn off of it with default settings.

Step 3

Now comes the fun part. Lets create /etc/NetworkManager/dnsmasq.d/00-default.conf and dump in our sexy content!

# Only match requests with a domain name in them
domain-needed

# Can't remember what this one did
bogus-priv

# Ignore /etc/resolv.conf
no-resolv

# Set a custom cache size (default is 400) for caching request responses
#cache-size=0

# Bind to the loopback network interface and my wifi card
interface=lo
interface=wlp2s0

# Poke all requests to a TLD of .test (e.g. my-app.test) to the localhost where my Caddy or Traefik servers are bound. We can list as many as we need.
address=/test/127.0.0.1

# Set the DNS servers to hit after failing to match to the above address
server=8.8.8.8
server=8.8.4.4

# Everyone likes logs, but maybe not right now
#log-queries
#log-facility=/tmp/dnsmasq.log

Step 4

Delete resolv.conf, this will force NetworkManager to rebuild it in our sexy image after we reboot in step 5.

$ sudo mv /etc/resolv.conf /etc/resolv.conf.bak

Step 5

Release the niffler right? WRONG. Now we reboot out PC. Yes we must. I know it sucks. And when we return everything should work and THEN and only THEN can we release the niffler.

And I can poke my web browser to planetside.test or super-awesome-secret-app.test without needing to mess with my /etc/hosts file every time.

This also means I don't have to modify the DNS settings for each different Wifi network I connect to to poke requests back to localhost, which is the main PITA.

Possible follow-ups

I'd like to see if I can get away with no specifying the server DNS addresses in the DNSMasq config and hold them in /etc/NetworkManager/conf.d/ with the global-dns-domain* options if possible.

I'd also like to build my own Traefik service in docker to manage my local dev projects, similar to how my work does it. Makes things very easy.

Update 20/01/2021

Added a step before reboot, we do have to remove resolve.conf entirely to force NetworkManager to rebuild it. Glory to the Niffler.

Update 20/01/2021 #2

Turns out the default firewall zone that Fedora imposes on the docker service blocks port 443, so composer install will fail to connect to Github over HTTPS/TLS. Had to install Red Hat's Firewall app and add port 443 to the exceptions for the docker0 connection zone "Ports" tabs.

@January 18, 2021