Tailscale Mesh VPN: Architecture, CGNAT, and Setup

In 2017, remote access in every office I managed ran through a single hub-and-spoke OpenVPN gateway. Every remote laptop had to establish an encrypted tunnel back to our central office router in Jakarta before it could reach an AWS database or a staging server in Singapore. When the office ISP line flickered on a Saturday afternoon, the whole remote workflow stopped. When fifty developers pulled large container images at the same time, the gateway CPU hit 100% and SSH sessions lagged for seconds.

Centralizing all traffic through one bastion box was the standard way to run remote access. We maintained dynamic DNS scripts, debugged asymmetric routing, and kept unreliable IPsec tunnels through NAT working.

Then mesh networking matured. WireGuard provided the encryption layer, and Tailscale packaged it into a product you can run without a central gateway.

When I first tested Tailscale between two nodes behind double Carrier-Grade NAT (CGNAT), it established a direct, peer-to-peer encrypted tunnel between them in three seconds, with zero port forwarding on either router.

This article covers how Tailscale works under the hood, how it gets through NAT, and how to set up a production mesh network.

The failure of legacy hub-and-spoke VPNs

To see what a mesh VPN solves, start with why traditional VPN architectures fail at scale.

In a traditional hub-and-spoke network (OpenVPN, IPsec, or WireGuard point-to-point):

Hub-and-Spoke Bottleneck:
Node A (Laptop in Tokyo) ──► Central VPN Gateway (Jakarta) ──► Node B (Server in Frankfurt)
                             (Single Point of Failure)

1. The Trombone Effect (Hairpinning): If a developer in Tokyo wants to send a file to a cloud instance in Frankfurt, the packets travel from Tokyo to Jakarta, get decrypted, inspected, re-encrypted, and then sent to Frankfurt. Latency increases sharply.
2. Centralized Bandwidth Choke: The central gateway must have enough upstream bandwidth to handle all inter-device communication.
3. Single Point of Failure: If the central VPN server crashes or its certificate expires, your entire team is locked out.

Tailscale removes the central concentrator. It builds a point-to-point mesh network (a tailnet).

Tailscale Peer-to-Peer Mesh:
Node A (Laptop in Tokyo) ══════[Direct WireGuard P2P]══════► Node B (Server in Frankfurt)

Traffic flows directly between nodes along the shortest physical internet path. If the control plane goes down, existing peer-to-peer connections keep running.

What it looks like vs what actually happened

When engineers first adopt Tailscale, they often mistake its simplicity for a cloud-hosted proxy. Here is what the surface behavior looks like versus what actually happens:

What it looks like on the surface	What actually happened under the hood
”Tailscale routes all my private traffic through their cloud”	Tailscale only coordinates metadata. Data packets flow directly P2P
”It bypasses firewalls using dangerous UPnP hacks”	It uses STUN-style endpoint discovery and UDP hole punching without touching router ports
”A WireGuard key leak compromises the entire fleet”	Every device pair negotiates distinct WireGuard keys, rotated continuously
”If internet fails, the whole mesh dies”	The control plane only distributes state. Existing tunnels survive offline
”Subnet routers expose everything blindly”	Granular ACLs and tag policies restrict traffic down to port and protocol level

These distinctions matter when you design a zero-trust network.

Under the hood: WireGuard, STUN, and DERP relays

Tailscale achieves its zero-configuration mesh through three core technologies:

1. WireGuard as the data plane engine

Tailscale does not invent a proprietary cryptographic protocol. It uses WireGuard, a modern VPN protocol built into the Linux kernel that relies on state-of-the-art cryptography: Curve25519 for key exchange, ChaCha20-Poly1305 for authenticated encryption, and BLAKE2s for hashing.

Unlike IPsec, which requires hundreds of lines of cryptographic negotiation, WireGuard behaves like SSH keys: every node has a public key and a private key. When Node A sends packets to Node B, it encrypts them directly with Node B’s public key.

2. NAT traversal and UDP hole punching

How do two devices behind residential NAT or mobile 4G/5G connections connect directly without port forwarding?

Tailscale performs NAT traversal with STUN-style endpoint discovery and coordinated UDP hole punching, following the same ideas as the ICE framework:

1. Discovery: Both nodes ask public STUN servers: “What is my public IP and external port mapping?”
2. Exchange: Nodes send their public endpoint candidates to the Tailscale coordination server (control plane).
3. Simultaneous UDP Hole Punching: Both nodes fire UDP packets toward each other’s discovered public endpoints at the exact same moment.
4. Stateful NAT Opening: Because Node A sent an outbound packet to Node B’s IP, Node A’s router opens a temporary state table entry allowing incoming packets from Node B. When Node B’s packet arrives, Node A’s router permits it through. A direct peer-to-peer connection is established.

3. DERP relays: the fallback when direct connections fail

What happens when you sit behind a symmetric NAT (common in hotels, enterprise firewalls, or cellular networks) that randomizes ports for every outgoing connection? UDP hole punching fails.

Instead of dropping the connection, Tailscale falls back to a DERP (Designated Encrypted Relay for Packets) node.

DERP nodes are globally distributed relay servers that communicate over standard HTTPS port 443.

Key security principle: The DERP server cannot decrypt your traffic. It only sees opaque WireGuard payloads. To the relay, your data looks like random bytes traveling over a TLS session. When the nodes can reach each other directly, Tailscale automatically upgrades back to a peer-to-peer connection.

You can verify connection status at any time:

tailscale status

Check active links:

tailscale ping 100.x.y.z

Output:

pong from node-b (100.85.12.34) via 198.51.100.12:41641 in 14ms (direct)

If it says via DERP(tokyo), the connection is relayed. If it says direct, packets are traveling straight across the internet. Results may vary in your environment.

Production setup: Linux node to mesh

Installing and connecting a Linux server takes less than sixty seconds.

Step 1: Install Tailscale

On Debian or Ubuntu hosts:

curl -fsSL https://tailscale.com/install.sh | sh

Step 2: Authenticate without a browser

On headless production servers, generate a reusable auth key from the Tailscale Admin Console and join with one command:

sudo tailscale up --authkey=tskey-auth-k1234567890-abcdef --ssh

The --ssh flag activates Tailscale SSH, allowing you to authenticate via your identity provider (Google Workspace, Okta, GitHub) without distributing, maintaining, or rotating static SSH public keys.

Step 3: Configure a subnet router (bridging private LANs)

You often have legacy equipment, printers, or staging databases where installing the Tailscale agent is impossible. A single Linux machine can act as a gateway into that physical subnet.

Enable IP forwarding in the Linux kernel:

echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
sudo sysctl -p /etc/sysctl.d/99-tailscale.conf

Advertise the private subnet route:

sudo tailscale up --advertise-routes=192.168.10.0/24

Approve the route in the Tailscale admin console. Every authorized device on your tailnet can then reach IP addresses in the 192.168.10.0/24 network as if they were on the same physical LAN.

Zero-trust access control with Tailscale ACLs

The worst way to run a mesh VPN is letting every laptop reach every production database.

Tailscale enforces network segmentation through declarative JSON/HuJSON Access Control Lists (ACLs). Instead of assigning rules to volatile IP addresses, you define rules based on cryptographic Tags and user identities.

Example production ACL policy:

{
  "tagOwners": {
    "tag:server": ["group:sysadmin"],
    "tag:database": ["group:sysadmin"]
  },
  "acls": [
    // Developers can access staging servers on port 80 and 443
    {
      "action": "accept",
      "src": ["group:developers"],
      "dst": ["tag:server:80,443"]
    },
    // Only sysadmins can SSH into production servers
    {
      "action": "accept",
      "src": ["group:sysadmin"],
      "dst": ["tag:server:22", "tag:database:5432"]
    }
  ]
}

If a developer’s laptop is compromised by malware, the attacker cannot pivot into production databases because the kernel drops the packets before they leave the virtual network adapter (tailscale0).

In our guide on Linux server hardening best practices, we emphasized defense in depth. Layering Tailscale ACLs on top of local firewall rules prevents lateral movement even if an internal host is breached.

Real field case: Connecting dual-region Proxmox clusters

A client operating two data centers (one in Singapore, one in Germany) needed their secondary Proxmox hypervisor to sync automated ZFS snapshots and backup volumes to an offsite node.

Neither location had a public static IP address. Both data centers were behind symmetric NAT firewalls, and the hosting facilities charged high monthly fees for public IP allocations.

Using traditional networking, this required renting a public cloud VPS, setting up an IPsec gateway, and routing hundreds of gigabytes through a slow, metered third-party relay.

Instead, we deployed Tailscale directly on both Proxmox Debian nodes. Within seconds, the Singapore node discovered the Frankfurt node. Traffic first flowed through a DERP relay while both sides probed each other’s endpoints, then upgraded to a direct WireGuard tunnel once UDP hole punching succeeded.

ZFS replication ran at the full line speed of their transit links, fully encrypted, with zero router ports opened. This worked because both networks allowed UDP hole punching. Whether you get a direct path or stay on a DERP relay depends on the NAT behavior at both ends, so run tailscale ping to confirm before relying on full line speed.

For deeper hypervisor security habits, check out our Proxmox security CVEs and patching guide and Proxmox Web UI access fix.

Summary checklist for production deployments

Before rolling out a mesh VPN across your infrastructure:

• Enable ip_forward on gateway nodes to support subnet routing.
• Implement Tailscale SSH to eliminate static SSH keys and enforce identity-backed MFA.
• Replace default “allow all” ACLs with tag-based least-privilege security policies.
• Run tailscale ping to confirm connections establish direct paths instead of staying on DERP relays.
• Enable device approval so unauthorized personal hardware cannot join your private tailnet unverified.

A centralized VPN gateway is no longer the only option for remote access. A mesh gives you direct encrypted paths between nodes, which is faster, simpler, and more resilient.

Have you migrated your homelab or production workloads to Tailscale yet? What has been your experience with NAT traversal and DERP fallbacks?