Aha! This was the last piece I was really looking for with WireGuard. It gets a bit tricky when you want packets to route between WireGuard clients. But once you grok how it works, well, it works.
This also works for passing traffic between WireGuard clients on the same interface – the trick is in making certain that AllowedIPs in the client configs includes the entire IP subnet services by the server, not just the single IP address of the server itself (with a /32 subnet)… and that you not only set up the tunnel on each client, but initialize it with a bit of data as well.
Set up your server with two WireGuard interfaces:
root@server:~# touch /etc/wireguard/keys/server.wg0.key root@server:~# chmod 600 /etc/wireguard/keys/server.wg0.key root@server:~# wg genkey > /etc/wireguard/keys/server.wg0.key root@server:~# wg pubkey < /etc/wireguard/keys/server.wg0.key > /etc/wireguard/keys/server.wg0.pub root@server:~# touch /etc/wireguard/keys/server.wg1.key root@server:~# chmod 600 /etc/wireguard/keys/server.wg1.key root@server:~# wg genkey > /etc/wireguard/keys/server.wg1.key root@server:~# wg pubkey < /etc/wireguard/keys/server.wg1.key > /etc/wireguard/keys/server.wg1.pub
Don’t forget to make sure ipv4 forwarding is enabled on your server:
root@server:~# sed -i 's/^#net\.ipv4\.ip_forward=1/net.ipv4.ip_forward=1/' /etc/sysctl.conf root@server:~# sysctl -p
Now set up your wg0.conf:
# server.wg0.conf [Interface] Address = 10.0.0.1/24 ListenPort = 51820 PrivateKey = WG0_SERVER_PRIVATE_KEY SaveConfig = false [Peer] # client1 PublicKey = PUBKEY_FROM_CLIENT_ONE AllowedIPs = 10.0.0.2/32
And your wg1.conf:
# server.wg1.conf [Interface] Address = 10.0.1.1/24 ListenPort = 51821 PrivateKey = WG1_SERVER_PRIVATE_KEY SaveConfig = false [Peer] # client2 PublicKey = PUBKEY_FROM_CLIENT_TWO AllowedIPs = 10.0.1.2/32
Gravy. Now enable both interfaces, and bring them online.
root@server:~# systemctl enable wg-quick@wg0 ; systemctl enable wg-quick@wg1 root@server:~# systemctl start wg-quick@wg0 ; systemctl start wg-quick@wg1
Server’s done. Now, set up your clients.
Client install, multi-wg server:
Client one will connect to the server’s wg0, and client two will connect to the server’s wg1. After creating your keys, set them up as follows:
# /etc/wireguard/wg0.conf on Client1 # connecting to server/wg0 [Interface] Address = 10.0.0.2/24 PrivateKey = PRIVATE_KEY_FROM_CLIENT1 # set up routing from server/wg0 to server/wg1 PostUp = route add -net 10.0.1.0/24 gw 10.0.0.1 ; ping -c1 10.0.0.1 PostDown = route delete -net 10.0.1.0/24 gw 10.0.0.1 SaveConfig = false [Peer] PublicKey = PUBKEY_FROM_SERVER AllowedIPs = 10.0.0.1/24, 10.0.1.1/24 Endpoint = wireguard.yourdomain.tld:51820
Now set up wg0 on Client2:
# /etc/wireguard/wg0.conf on Client2 # connecting to server/wg1 [Interface] Address = 10.0.1.2/24 PrivateKey = PRIVATE_KEY_FROM_CLIENT2 # set up routing from server/wg1 to server/wg0 PostUp = route add -net 10.0.0.0/24 gw 10.0.1.1 ; ping -c1 10.0.1.1 PostDown = route delete -net 10.0.0.0/24 gw 10.0.1.1 SaveConfig = false [Peer] PublicKey = PUBKEY_FROM_SERVER AllowedIPs = 10.0.0.1/24, 10.0.1.1/24 Endpoint = wireguard.yourdomain.tld:51821
Now start up the client interfaces. First, Client1:
root@client1:~# systemctl start wg-quick@wg0 root@client1:~# wg interface: wg0 public key: MY_PUBLIC_KEY private key: (hidden) listening port: some-random-port peer: SERVER_PUBLIC_KEY endpoint: server-ip-address:51820 allowed ips: 10.0.0.0/24, 10.0.1.0/24 latest handshake: 3 seconds ago transfer: 2.62 KiB received, 3.05 KiB sent
Now Client2:
root@client2:~# systemctl start wg-quick@wg0 root@client2:~# wg interface: wg0 public key: MY_PUBLIC_KEY private key: (hidden) listening port: some-random-port peer: SERVER_PUBLIC_KEY endpoint: server-ip-address:51821 allowed ips: 10.0.0.0/24, 10.0.1.0/24 latest handshake: 4 seconds ago transfer: 2.62 KiB received, 3.05 KiB sent
Now that both clients are connected, we can successfully send traffic back and forth between client1 and client2.
root@client1:~# ping -c3 10.0.1.2 PING 10.0.1.2 (10.0.1.2) 56(84) bytes of data. 64 bytes from 10.0.1.2: icmp_seq=1 ttl=63 time=80.4 ms 64 bytes from 10.0.1.2: icmp_seq=2 ttl=63 time=83.5 ms 64 bytes from 10.0.1.2: icmp_seq=3 ttl=63 time=83.2 ms --- 10.0.1.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 80.492/82.410/83.501/1.380 ms
Sweet.
The step some of you undoubtedly missed:
There’s a little trick that will frustrate you to no end if you glossed over the client config files without paying attention.
PostUp = route add -net 10.0.0.0/24 gw 10.0.1.1 ; ping -c1 10.0.1.1
If you left out that ping command in one or both of the clients, then you won’t be able to ping client1 from client2, or vice versa, until each client has sent some data down the tunnel. This frustrated the crap out of me initially – ping client2 from client1 would be broken, but then ping client1 from client2 would work and then ping client2 from client1 would work. The solution, as shown, is just to make sure we immediately chuck a packet down the tunnel whenever each client connects; then everything works as intended.
What if I don’t want multiple interfaces?
If all you want to do is pass traffic from one client to the next, you don’t need two interfaces on the server, and you don’t need PostUp and PostDown route commands, either. The trick is that you do need to make sure that AllowedIPs on each client is set to a range that includes the other client, and you do need the PostUp ping -c1 server-ip-addresscommand on each client as well. Without that PostUp ping, you’re going to get frustrated by connectivity that “sometimes works and sometimes doesn’t”.
If you want to give access to some clients but not all clients, you can do that by setting multiple AllowedIPs arguments on the clients, like so:
[Peer]
PublicKey = PUBKEY_FROM_SERVER
# this stanza allows access from the server (.1), client one (.2),
# and client two (.3) - but not from any clients at .4-.254.
AllowedIPs = 10.0.0.1/32, 10.0.0.2/32, 10.0.0.3/32
Endpoint = wireguard.yourdomain.tld:51820
That is a sample [Peer] stanza of a client wg config, not a[Peer] stanza of the server wg config! The[Peer] stanzas of the server config should only allow connection to a single IP (using a /32 subnet) for each individual[Peer] definition.
If you try to set AllowedIPs 10.0.0.0/24 on both client1 and client2’s[Peer] stanzas in the server’s wg config, you’ll break one or the other client – they can’t BOTH be allowed the entire subnet.
This bit’s a little confusing, I know, but that’s how it works.
As always, caveat imperator.
Just like the last couple of posts, everything here is tested and working. Also just like the last couple of posts, this is day zero of my personal experience with Wireguard. While everything here works, I still have not delved into any of the actual crypto components’ configurability or personally evaluated/researched how sane the defaults are.
These configs absolutely will encrypt the data sent down the tunnel – I’ve verified that much! – but I cannot offer a well-established and researched opinion on how well they encrypt that data, or what weaknesses there might be. I have no particular reason to believe they’re not secure, but I haven’t done any donkey-work to thoroughly establish that they are secure either.
Caveat imperator.
I may come back to edit out these dire warnings in a few weeks or months as I hammer at this stuff more. Or I might forget to! Feel free to tweet @jrssnet if you want to ask how or if things have changed; I am definitely going to continue my evaluation and testing.
nice, clean write up!
i‘ve tried to route packets from wg clients/servers to a non wg subnet behind a single wg client.
man, i’ve routed, masqueraded, forwarded… even cross-compiled socks5 servers in a moment of rage ….. these packets wouldn‘t route.
well… after reading your piece i am mildly optimistic for another round 🙂 …damnit
Nice write up… Site-to-site version would be good to see next. I too am battling with routing to entire subnets behind wg. This stuff is a black art / hell by trial and error!
+1 for your labour though!
One thing not very obvious to new comers to add : WG machines & networks notation ARE NOT CIDR, so also do not care the logs complaining about bad CIDR notation too. Many people fall into this trap and miss their setup for this reason.
Very nice and clear article, what I didn’t find anywhere else.
Have you thought about another scenario? Where you want to have two completely isolated subnets, each with its own clients that can communicate within the subnet but can’t reach the clients on the other subnet.
I know if you don’t include that routes to the client, the goal is reached, but as security aspect, it is better to do it on the server. So the clients can’t escalate their access just by adding a static route to the routing table.
Regards
Arman
There is a lot of misunderstandings of how Wireguard routes packets via the AllowedIPs settings. The Wireguard docs are lacking in specifics. Worse, some of the documentation out there is just plain wrong. Your example is one of the better ones.
My understanding after beating on a Cloud VPN relay server is that the allowed ips is what is ALLOWED INTO the VPN pipe from that interface. Hence, if at a client you enter 192.168.1.3, then any packets originating from that IP address will be allowed INTO the pipe. The same thinking works at the “server”.
If you look at the routing table on the client after that client is setup and running , it should have added the 192.168.1.3 to the wg device on the client. Reading the routing table from the bottom up; when the 192.168.1.3 ip is encountered, you should see wg_ device (or the wg IP address depending on your os) on the right side. When the wg interface on your client is deactivated, that route should disappear from the routing table. This works properly on the windows and linux clients. Yes, I know everything is called “peers” (poor nomenclature!), but one device initiates the connection, so I consider that device to be a client.
Regarding the need to ping the other device on a PostUp, I have not seen that need as of 2020. There are still a lot of growing pains with Wireguard. FYI – Wireguard won’t work on a Raspberry Pi without Systemd-Networkd acting as the network manager.
If you get a chance, I’d like to see you up this with your current experiences! This is one of he better blogs on Wireguard!
Hello,
Thanks for this article. I struggled for 2 days with the AllowedIp setting !
Would’nt a PersistentKeepalive be better than the PostUp ping ?
Hello, many thanks for this post.
I should point out that the correct quote is “caveat emptor” and not “caveat imperator.”
I tested the ping in my wg-quick
since I plan using a mesh configuration I put in two different address, `ping -c1 x.x.x.x` but if I had one member of the mesh down then after timing out the single ping I got a ip link delete dev wg0.
“caveat emptor” means “buyer beware.”
“caveat imperator”–in context–means “administrator beware.”
The PostUp ping is there specifically to enable a ping from the “server” (the peer with a publicly-reachable Listen endpoint) to the “client” (the peer without a public endpoint, which reaches out to the “server” peer).
If you don’t need to be able to initiate communication with “client” peers–eg, you only need THEM to be able to initiate communications to stuff on the other side–you don’t need the PostUp ping. But without that command, “client” peers will only be reachable if they have recent network activity since the tunnel came up.