Internal DNS with NPM and PiHole and how to keep Cloudflare Tunnels from breaking

Internal DNS with NPM and PiHole and how to keep Cloudflare Tunnels from breaking

For a long time to access a docker app on my home server, I had to remember which random port I assigned to it. 192.168.1.50:8080 for one app, port 3010 for another, and port 5678 for a third. It worked, but it was messy, hard to remember, and lacked a cohesive security layer.

Recently, I decided it was time to do things the “right” way. I set out to move my Docker containers behind a proper NGINX Proxy Manager (NPM) instance, secure them with a wildcard SSL certificate, and manage it all via Pihole for local DNS. These individual container are now also NOT exposed to my host anymore, NPM manages all the reverse proxying.

I also needed to ensure my Cloudflare Tunnels continued to provide secure remote access.

In this post, I’ll walk you through how I migrated to proper isolated containers and how I moved them behind a centralized “Internal Proxy” network, solved the dreaded “502 Bad Gateway” Cloudflare tunnel errors, and finally ditched the random port numbers for good.

Portainer Network Setup

The first step is to create a network in Portainer. Set your IP ranges as you need them. I called my network “internal_proxy“.

I will put NPM and all the containers I want to go via this reverse proxy inside this network. Cloudflared, which I use for tunneling, will live in 2 networks, more on this later.

PiHole DNS Setup

I decided to use “internal.leighonline.net” as my domain because I already own the domain “leighonline.net“.

I don’t want to add every single DNS record, so we will use a wildcard so that ANYTHING with the domain “xxxx.internal.leighonline.net” resolves to our docker host.

In order to accomplish this, go to Settings -> All Settings -> Miscellaneous and add this dnsmasq entry. Change the IP address to your IP.

NGINX Proxy Manager and the SSL Cert

Next we can set up NPM (NGINX Proxy Manager) in docker and get an SSL cert for our domain. Note that NPM lives inside my “internal_proxy” network.

services:
  nginxproxymanager:
    container_name: nginxproxymanager
    networks:
      - internal_proxy
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      - '80:80'    # Public HTTP Port
      - '443:443'  # Public HTTPS Port
      - '81:81'    # Admin Web Port
    environment:
      DISABLE_IPV6: 'true'
    volumes:
      - /opt/nginxproxymanager:/data
      - /opt/nginxproxymanager:/etc/letsencrypt

networks:
  internal_proxy:
    external: true # We created this network in portainer manually

Once your container is up and running, go to http://192.168.1.109:81 which is your admin interface and generate a new SSL Certificate and choose DNS Authentication.

Take note of the *. in front of the domain name, this will get you a wildcard certificate which means ANYTHING on internal.leighonline.net can use this certificate.

To generate a Token in Cloudflare, go here. You just need to give that token Zone permissions to the domain name you will be using. (Cloudflare calls a domain name a Zone).

Once you have the token, paste it into your NPM window where it asks for the token and wait a few minutes. It will look something like this when the certificate has been generated. Your status will be “Not Used” or the like because no domains are associated with it yet.

Back in Portainer

Next we need to change our existing docker stacks to:

• Be inside this new network we created in step 1. This will allow NPM to forward traffic to these containers using the service name.
• Remove the “ports” directive so these containers are NOT exposed to the host anymore. NPM will handle all connections to these apps now. Our containers are now isolated.

Here is my stirlingpdf stack as an example:

services:
  stirlingpdf:
    image: stirlingtools/stirling-pdf:latest
    container_name: stirlingpdf_app
    networks:
      - internal_proxy
    # ports:
    #  - '8040:8080'
    expose: # Just informative
      - 8080
    volumes:
      - /opt/stirlingpdf/trainingData:/usr/share/tessdata # Required for extra OCR languages
      - /opt/stirlingpdf/extraConfigs:/configs
      - /opt/stirlingpdf/customFiles:/customFiles/
      - /opt/stirlingpdf/logs:/logs/
      - /opt/stirlingpdf/pipeline:/pipeline/
    environment:
      - DOCKER_ENABLE_SECURITY=true
      - SECURITY_ENABLE_LOGIN=true
      - LANGS=en_GB

networks:
  internal_proxy:
    external: true

As you can see I commented out ports and added this stack into my internal_network. This container is now not exposed to the host anymore. NPM will handle connections to it.

Back in NPM

Now lets add a new Proxy Host:

• Set your preferred domain name on “internal.leighonline.net”. I chose “stirlingpdf.internal.leighonline.net”.
• The Forward Hostname/IP is the docker compose service name we defined up top.
• The port is the INTERNAL port that this container is listening on. You can find this on the applications docker or github page, or by looking at the docker file.

Under the SSL Tab, select your newly created SSL certificate from the dropdown and enable these 2 options:

Testing Time

Type this command:

nslookup stirlingpdf.internal.leighonline.net

You will see it resolves via your Pihole:

You will also see that ANYTHING under internal.leighonline.net will resolve, because we told Pihole to do so a few sections up when we added that dnsmasq config:

Cloudflare Tunnel Setup

We need to make sure our Cloudflare tunnels still work. Change your cloudflared compose (stack) file to be in both the internal network and in its default network.

By adding it to both, cloudflared can get to things outside of our new network as well, like anything else on your network such as Plex, your Sonic Pad, etc.

services:
  cloudflared:
    image: cloudflare/cloudflared:latest
    container_name: cloudflared
    restart: unless-stopped
    command: tunnel run --token eyJhxxxxxxxxxxx
    networks:
      - default          # This keeps your LAN access (Sonic Pad, etc.)
      - internal_proxy   # This adds access to your new network
      
networks:
  internal_proxy:
    external: true

Look at the orange block in the image below. Instead of pointing your applications to the container’s IP and port, you now just give it the NPM service name. (the service name is defined in your compose file)

For each app though, we need to add host headers. Because each app now just points to our NPM, our NPM needs to know which host to pass the request and the host headers take care of this.

So add “stirlingpdf.internal.leighonline.net” in both the TLS Origin Server Header and the HTTP Host Header fields. This has to do with SNI and all that.