amnezia-vpn/amnezia-client

Fork 0

mirror of https://github.com/amnezia-vpn/amnezia-client.git synced 2026-10-13 21:33:09 +00:00

[GH-ISSUE #3090] Linux 5.0.1.5: exclusion routes never installed — netlink route dump overflows the fixed 8 KiB buffer #1431

New issue

Open

opened 2026-10-13 20:42:20 +00:00 by bsakaguchi · 8 comments

bsakaguchi commented

2026-10-13 20:42:20 +00:00

Owner

Originally created by @pers1307 on GitHub (Aug 29, 2026).
Original GitHub issue: https://github.com/amnezia-vpn/amnezia-client/issues/3090

Describe the bug

On Linux, client 5.0.1.5 fails to read the routing table and never installs
exclusion routes for the VPN endpoint. With 0.0.0.0/1 and 128.0.0.0/1
pointing at the tunnel, WireGuard packets addressed to the server are routed
back into the tunnel itself, creating a routing loop. Packets grow with each
iteration until the socket rejects them with sendmmsg: message too long.

Client 4.8.21 on the very same machine installs the routes correctly and works.
The issue is protocol-independent — reproduced with both AmneziaWG 2.0 and 3.1
servers.

To Reproduce

Steps to reproduce the behavior:

Install 5.0.1.5 on Ubuntu 24.04 with a normal DHCP default route
Connect to any AmneziaWG server
Run ip route get <server-ip> — it points at the tunnel (dev amn0) instead
of the physical gateway
See sendmmsg: message too long in the daemon log; no traffic passes

Expected behavior

The daemon should add an exclusion route for the server endpoint through the
physical default gateway, as 4.8.21 does:

<server-ip> via 192.168.31.1 dev wlp0s20f3

Log files

Same 15-second window, two client versions on one machine:

Log line	5.0.1.5	4.8.21
`Error in received packet: Numerical argument out of domain`	10	0
`No default gateway available, skipping exclusion route`	10	0
exclusion routes actually installed	none	all

5.0.1.5:

LinuxRouteMonitor : Adding exclusion route for <server-ip>/32
Error in received packet: No such file or directory
LinuxRouteMonitor : No default gateway available, skipping exclusion route
...
WireguardGo : ERROR: (amn0) peer(XXXX…) - Failed to send data packets:
              write udp 0.0.0.0:54793: sendmmsg: message too long

4.8.21, same step:

LinuxRouteMonitor : Adding exclusion route for <server-ip>/32

(no warning, route present in ip route)

Other errors present in both versions, possibly related:

Error: argument "amnvpnrt" is wrong: table id value is invalid
mkdir: cannot create directory '/sys/fs/cgroup/net_cls/amnvpnexclusions/'
LinuxRouteMonitor : Adding exclusion route for /999999 → Invalid destination prefix
net_cls cgroup v1 not available, traffic splitting disabled

Screenshots

N/A — the client reports "Connected".

Desktop (please complete the following information):

OS: Ubuntu 24.04.4 LTS (kernel 6.17), KDE Plasma / X11
Version: 5.0.1.5 (regression — 4.8.21 works)

Smartphone (please complete the following information):

Not affected — mobile clients handle routing through the OS VPN API

Server (please complete the following information):

OS: Ubuntu 24.04, Docker

Additional context

Workaround — add the exclusion route manually before connecting:

sudo ip route add <server-ip> via <lan-gateway> dev <iface>

The loop disappears and sendmmsg: message too long stops.

Possibly related: #1498, #1357, #2048 — all report "connected but no internet"
on Linux with the same amnvpnrt / Invalid destination prefix log noise.

Originally created by @pers1307 on GitHub (Aug 29, 2026). Original GitHub issue: https://github.com/amnezia-vpn/amnezia-client/issues/3090 **Describe the bug** On Linux, client 5.0.1.5 fails to read the routing table and never installs exclusion routes for the VPN endpoint. With `0.0.0.0/1` and `128.0.0.0/1` pointing at the tunnel, WireGuard packets addressed to the server are routed back into the tunnel itself, creating a routing loop. Packets grow with each iteration until the socket rejects them with `sendmmsg: message too long`. Client 4.8.21 on the very same machine installs the routes correctly and works. The issue is protocol-independent — reproduced with both AmneziaWG 2.0 and 3.1 servers. **To Reproduce** Steps to reproduce the behavior: 1. Install 5.0.1.5 on Ubuntu 24.04 with a normal DHCP default route 2. Connect to any AmneziaWG server 3. Run `ip route get <server-ip>` — it points at the tunnel (`dev amn0`) instead of the physical gateway 4. See `sendmmsg: message too long` in the daemon log; no traffic passes **Expected behavior** The daemon should add an exclusion route for the server endpoint through the physical default gateway, as 4.8.21 does: ``` <server-ip> via 192.168.31.1 dev wlp0s20f3 ``` **Log files** Same 15-second window, two client versions on one machine: | Log line | 5.0.1.5 | 4.8.21 | |---|---|---| | `Error in received packet: Numerical argument out of domain` | 10 | 0 | | `No default gateway available, skipping exclusion route` | 10 | 0 | | exclusion routes actually installed | none | all | 5.0.1.5: ``` LinuxRouteMonitor : Adding exclusion route for <server-ip>/32 Error in received packet: No such file or directory LinuxRouteMonitor : No default gateway available, skipping exclusion route ... WireguardGo : ERROR: (amn0) peer(XXXX…) - Failed to send data packets: write udp 0.0.0.0:54793: sendmmsg: message too long ``` 4.8.21, same step: ``` LinuxRouteMonitor : Adding exclusion route for <server-ip>/32 ``` (no warning, route present in `ip route`) Other errors present in both versions, possibly related: ``` Error: argument "amnvpnrt" is wrong: table id value is invalid mkdir: cannot create directory '/sys/fs/cgroup/net_cls/amnvpnexclusions/' LinuxRouteMonitor : Adding exclusion route for /999999 → Invalid destination prefix net_cls cgroup v1 not available, traffic splitting disabled ``` **Screenshots** N/A — the client reports "Connected". **Desktop (please complete the following information):** - OS: Ubuntu 24.04.4 LTS (kernel 6.17), KDE Plasma / X11 - Version: 5.0.1.5 (regression — 4.8.21 works) **Smartphone (please complete the following information):** - Not affected — mobile clients handle routing through the OS VPN API **Server (please complete the following information):** - OS: Ubuntu 24.04, Docker **Additional context** Workaround — add the exclusion route manually before connecting: ``` sudo ip route add <server-ip> via <lan-gateway> dev <iface> ``` The loop disappears and `sendmmsg: message too long` stops. Possibly related: #1498, #1357, #2048 — all report "connected but no internet" on Linux with the same `amnvpnrt` / `Invalid destination prefix` log noise.

bsakaguchi commented

2026-10-13 20:42:21 +00:00

Author

Owner

@pers1307 commented on GitHub (Aug 30, 2026):

Confirmed — this is exactly my case.

$ ip -4 route show table all | wc -l
60
$ ip -6 route show table all | wc -l
170

230 entries combined, roughly 2.4× over the ~94 threshold you mention
(230 × ~87 bytes ≈ 19.5 KiB against the 8 KiB buffer).

The bulk comes from Docker — 15 networks and 36 running containers on this
machine, accounting for 162 of the IPv6 routes and 39 of the IPv4 ones. The
tunnel itself contributes only 3 entries, so the overflow is there before the
VPN even starts.

That also explains the inconsistent perror text I saw: the daemon logged
Numerical argument out of domain on one connection attempt and
No such file or directory on another, within the same session — which never
made sense as a real error.

And it matches the version split I observed: on 4.8.21 the exclusion route is
installed correctly and everything works, while 5.0.1.5 logs
No default gateway available, skipping exclusion route 10 times per attempt
and ends with sendmmsg: message too long. Same machine, same server, minutes
apart.

Happy to test #3052 against this setup if a build is available — the route
table here reproduces the overflow reliably.

@pers1307 commented on GitHub (Aug 30, 2026): Confirmed — this is exactly my case. ``` $ ip -4 route show table all | wc -l 60 $ ip -6 route show table all | wc -l 170 ``` 230 entries combined, roughly 2.4× over the ~94 threshold you mention (230 × ~87 bytes ≈ 19.5 KiB against the 8 KiB buffer). The bulk comes from Docker — 15 networks and 36 running containers on this machine, accounting for 162 of the IPv6 routes and 39 of the IPv4 ones. The tunnel itself contributes only 3 entries, so the overflow is there before the VPN even starts. That also explains the inconsistent `perror` text I saw: the daemon logged `Numerical argument out of domain` on one connection attempt and `No such file or directory` on another, within the same session — which never made sense as a real error. And it matches the version split I observed: on 4.8.21 the exclusion route is installed correctly and everything works, while 5.0.1.5 logs `No default gateway available, skipping exclusion route` 10 times per attempt and ends with `sendmmsg: message too long`. Same machine, same server, minutes apart. Happy to test #3052 against this setup if a build is available — the route table here reproduces the overflow reliably.

bsakaguchi commented

2026-10-13 20:42:21 +00:00

Author

Owner

@HeisLuka commented on GitHub (Sep 9, 2026):

I rechecked this against current dev.

The reported root cause is still present, and I think there are several additional failure boundaries worth fixing together with #3052.

1. The fixed 8 KiB netlink dump buffer is still in current dev

NetworkUtilities::getGatewayAndIface() currently allocates:

constexpr int BUFFER_SIZE = 8192;

char msgbuf[100];
char buffer[BUFFER_SIZE];
char *ptr = buffer;

and accumulates the entire multipart RTM_GETROUTE dump into that one buffer:

received_bytes =
    recv(sock,
         ptr,
         sizeof(buffer) - msg_len,
         0);

After each datagram:

ptr += received_bytes;
msg_len += received_bytes;

So once:

msg_len == 8192

the next call effectively becomes:

recv(sock, ptr, 0, 0);

which returns 0.

The code only treats:

received_bytes < 0

as a receive error.

It then executes:

NLMSG_OK(nlh, received_bytes)

with:

received_bytes = 0

which fails and reaches:

perror("Error in received packet");
return {};

So the varying:

No such file or directory
Numerical argument out of domain

messages are indeed unrelated stale errno text.

The actual invariant failure is:

route dump > 8192 bytes
        ↓
remaining receive capacity becomes 0
        ↓
gateway lookup returns empty

The reporter's measured:

IPv4 routes:  60
IPv6 routes: 170

total: 230

makes this deterministic rather than timing-dependent.

2. The request unnecessarily asks for both IPv4 and IPv6 routes

The rtmsg request buffer is zero-initialized, but current code never assigns:

route_entry->rtm_family

for the request.

So:

rtm_family = AF_UNSPEC

and the dump includes both IPv4 and IPv6.

That makes the overflow substantially easier to reach on Docker/libvirt/container hosts.

It is especially inconsistent because the parser later interprets every RTA_GATEWAY using:

inet_ntop(AF_INET, ...)

So the function is semantically an IPv4 gateway lookup while requesting an IPv4+IPv6 route dump.

#3052 correctly narrows the request to:

request.rtm.rtm_family = AF_INET;

3. The current parser does not actually require a default route

After receiving the dump, current code accepts the first RT_TABLE_MAIN route containing both:

RTA_GATEWAY
RTA_OIF

There is no check for:

rtm_dst_len == 0

and route metric is not considered.

Therefore:

gateway-bearing route
!=
default route

but getGatewayAndIface() currently treats them as equivalent.

A host with multiple gateways or more-specific routes can therefore return the wrong underlay even when the dump fits into 8192 bytes.

For a VPN client the intended lookup is specifically:

best IPv4 default route

not:

first main-table route with a gateway

The rtm_dst_len == 0 and lowest-metric selection in #3052 address this correctly.

4. More importantly: failure to install the endpoint exclusion is ignored

This is the part I would treat as the critical lifecycle bug.

During Daemon::activate() current code does:

for (const QString& i : config.m_excludedAddresses) {
    addExclusionRoute(IPAddress(i));
}

The return value is ignored.

So:

endpoint exclusion failed

does not stop activation.

The peer is then configured:

if (!wgutils()->updatePeer(config)) {
    ...
}

and finally the full-tunnel routes are installed.

For IPv4 /0, Linux intentionally converts:

0.0.0.0/0

into:

0.0.0.0/1
128.0.0.0/1

pointing to amn0.

Therefore the actual sequence can be:

try endpoint /32 exclusion
        ↓
getGatewayAndIface() overflows
        ↓
"No default gateway available"
        ↓
exclusion NOT installed
        ↓
error ignored
        ↓
peer configured
        ↓
0.0.0.0/1 + 128.0.0.0/1 installed
        ↓
VPN endpoint now matches a tunnel route

At that point the client has knowingly committed a full-tunnel route while its control-plane escape route is absent.

That should be a forbidden state.

5. `updatePeer()` repeats the same failed operation and also ignores it

WireguardUtilsLinux::updatePeer() contains another endpoint-exclusion attempt:

if ((config.m_hopType != InterfaceConfig::MultiHopExit) &&
    (m_rtmonitor != nullptr)) {

    m_rtmonitor->addExclusionRoute(
        IPAddress(config.m_serverIpv4AddrIn));

    m_rtmonitor->addExclusionRoute(
        IPAddress(config.m_serverIpv6AddrIn));
}

Again the return values are ignored.

So there are currently two independent best-effort layers:

Daemon::activate()
    -> exclusion failure ignored

WireguardUtilsLinux::updatePeer()
    -> exclusion failure ignored

Neither establishes:

VPN endpoint is reachable through underlay

before installing full-tunnel routes.

I would consolidate ownership of this operation rather than attempting it twice.

6. This explains the recursive WireGuard packet growth

If the server is, for example:

203.0.113.50

and its physical /32 exclusion route is missing, one of:

0.0.0.0/1
128.0.0.0/1

wins.

The userspace WireGuard process attempts to send its outer UDP packet to:

203.0.113.50

but the kernel routes that packet into:

amn0

The packet presented to the tunnel is then treated as traffic that itself needs tunnelling.

Conceptually:

inner packet
    ↓
WG encapsulation
    ↓
outer UDP to VPN server
    ↓
route says amn0
    ↓
outer packet becomes tunnel input
    ↓
WG encapsulates it again
    ↓
larger outer packet
    ↓
amn0 again
    ↓
...

until packet size limits are reached.

That makes:

sendmmsg: message too long

a very plausible downstream consequence of the missing endpoint escape route, rather than an independent MTU problem.

The user's manual workaround:

ip route add <server-ip> \
    via <physical-gateway> \
    dev <physical-interface>

breaking the loop is a strong runtime confirmation of the same path.

7. The error path also leaks a netlink socket

getGatewayAndIface() creates:

sock = socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE)

but close(sock) only occurs on the normal path near the end of the function.

For example the overflow path reaches:

perror("Error in received packet");
return {};

without closing sock.

The same happens for several other early returns.

Therefore every failed gateway lookup can leak one netlink FD.

The reporter sees approximately ten gateway lookup failures per connection attempt.

So repeated reconnects can additionally produce:

gateway lookup failure
    ↓
netlink FD leaked
    ↓
retry
    ↓
another FD leaked

This should probably use an RAII/scoped FD rather than manually trying to cover every return path.

8. #2862 explains the apparent 4.8.x -> 5.0.x regression

PR #2862 changed this code from:

100-byte buffer
+
incorrect multipart handling

to:

8192-byte buffer
+
proper response-header multipart handling

and changed parsing from approximately the first receive block to the accumulated dump.

So older releases could accidentally succeed because they effectively parsed only an early route message.

Once multipart reception was corrected, large route tables started exposing the fixed 8192-byte ceiling.

So I would not treat:

4.8.21 works

as evidence that the previous implementation was correct.

It happened to avoid this failure mode.

9. #3052 fixes the main parser bug, but activation should also enforce the endpoint-route invariant

I think #3052 is the right direction for the route parser:

AF_INET-only dump
parse multipart responses incrementally
require actual default route
respect route metric

But even with the parser fixed, I would separately harden the activation transaction.

For a single-hop/full-tunnel connection:

endpoint exclusion

is not optional configuration.

It is a prerequisite for making tunnel routes authoritative.

The ordering should be approximately:

resolve endpoint
        ↓
resolve usable underlay route
        ↓
install endpoint exclusion
        ↓
verify endpoint best route is NOT amn0
        ↓
configure peer
        ↓
install /1 tunnel routes
        ↓
verify endpoint still resolves to underlay
        ↓
continue connection

If the exclusion cannot be established:

activation must fail

rather than:

warning
+
continue into a routing loop

10. Distinguish parser failure from "no default route yet"

There is also an important distinction with cases such as boot-time autoconnect.

These are different states:

A. RTM_GETROUTE failed / malformed / truncated

B. route dump succeeded,
   but no IPv4 default route exists yet

Both currently collapse into:

gateway.isEmpty()

and then:

No default gateway available

But the recovery policy is different.

For A:

internal routing discovery failure

For B:

underlay not ready yet
-> bounded retry / wait for route event

It would be useful for getGatewayAndIface() to return a typed result instead of encoding all failure classes as an empty QPair.

11. IPv6 endpoint exclusion is another adjacent problem

updatePeer() attempts both:

serverIpv4AddrIn
serverIpv6AddrIn

but Linux rtmSendRoute() obtains the gateway through the IPv4-only:

getGatewayAndIface()

and always creates the gateway attribute from:

struct in_addr ip4;
inet_pton(AF_INET, gateway.toUtf8(), &ip4);

So the current exclusion construction is not genuinely family-aware.

That should probably be separated from #3090, but a future route lookup API should return something like:

family
gateway
interface
metric

rather than a QString IPv4 gateway plus QNetworkInterface.

Suggested invariants

The most important one is:

FULL_TUNNEL_ROUTE_COMMITTED
    =>
VPN_ENDPOINT_ROUTE_CONFIRMED_ON_UNDERLAY

And for route discovery:

successful default-gateway lookup
    =>
complete netlink transaction
AND actual default route
AND correct address family
AND selected route policy is deterministic

Regression tests

More than 8 KiB of route dump data:

Docker / libvirt / many interfaces

Expected:

default gateway still found

More than 16 KiB / much larger dumps.

The fix should not merely move the hard ceiling.

Large IPv6 table + small IPv4 table:

IPv4 default lookup

must not process unrelated IPv6 routes.

Multiple IPv4 default routes with different metrics:

expected = correct best default

Specific gateway route appears before the default route.

Expected:

must not be selected as default

Force route-discovery failure while connecting full tunnel.

Expected:

activation fails before /1 routes are committed

Forbidden:

warning + continue

Default route appears shortly after boot.

Expected:

bounded readiness/retry

rather than either an infinite retry or committing a broken tunnel.

Repeated failed gateway lookups.

Expected:

no netlink FD growth

Full tunnel after successful setup:

ip route get <VPN endpoint>

must return the physical underlay interface.

At the same time:

ip route get <ordinary public IP>

must return amn0.

Deliberately remove the endpoint exclusion after startup.

The connection-health logic should detect loss of the control-plane route rather than silently entering recursive encapsulation.

So I would classify #3090 as confirmed and still present in current dev.

The fixed 8 KiB netlink buffer is the immediate regression trigger on the reported host, but the more important lifecycle problem is broader:

failure to establish the VPN endpoint's underlay route is treated as best-effort, and full-tunnel routes are committed anyway.

That turns a recoverable route-discovery failure into a self-capturing VPN endpoint and complete loss of connectivity.

@HeisLuka commented on GitHub (Sep 9, 2026): I rechecked this against current `dev`. The reported root cause is still present, and I think there are several additional failure boundaries worth fixing together with #3052. ## 1. The fixed 8 KiB netlink dump buffer is still in current dev `NetworkUtilities::getGatewayAndIface()` currently allocates: ```cpp constexpr int BUFFER_SIZE = 8192; char msgbuf[100]; char buffer[BUFFER_SIZE]; char *ptr = buffer; ``` and accumulates the entire multipart `RTM_GETROUTE` dump into that one buffer: ```cpp received_bytes = recv(sock, ptr, sizeof(buffer) - msg_len, 0); ``` After each datagram: ```cpp ptr += received_bytes; msg_len += received_bytes; ``` So once: ```text msg_len == 8192 ``` the next call effectively becomes: ```cpp recv(sock, ptr, 0, 0); ``` which returns `0`. The code only treats: ```cpp received_bytes < 0 ``` as a receive error. It then executes: ```cpp NLMSG_OK(nlh, received_bytes) ``` with: ```text received_bytes = 0 ``` which fails and reaches: ```cpp perror("Error in received packet"); return {}; ``` So the varying: ```text No such file or directory Numerical argument out of domain ``` messages are indeed unrelated stale `errno` text. The actual invariant failure is: ```text route dump > 8192 bytes ↓ remaining receive capacity becomes 0 ↓ gateway lookup returns empty ``` The reporter's measured: ```text IPv4 routes: 60 IPv6 routes: 170 total: 230 ``` makes this deterministic rather than timing-dependent. ## 2. The request unnecessarily asks for both IPv4 and IPv6 routes The `rtmsg` request buffer is zero-initialized, but current code never assigns: ```cpp route_entry->rtm_family ``` for the request. So: ```text rtm_family = AF_UNSPEC ``` and the dump includes both IPv4 and IPv6. That makes the overflow substantially easier to reach on Docker/libvirt/container hosts. It is especially inconsistent because the parser later interprets every `RTA_GATEWAY` using: ```cpp inet_ntop(AF_INET, ...) ``` So the function is semantically an IPv4 gateway lookup while requesting an IPv4+IPv6 route dump. `#3052` correctly narrows the request to: ```cpp request.rtm.rtm_family = AF_INET; ``` ## 3. The current parser does not actually require a default route After receiving the dump, current code accepts the first `RT_TABLE_MAIN` route containing both: ```text RTA_GATEWAY RTA_OIF ``` There is no check for: ```cpp rtm_dst_len == 0 ``` and route metric is not considered. Therefore: ```text gateway-bearing route != default route ``` but `getGatewayAndIface()` currently treats them as equivalent. A host with multiple gateways or more-specific routes can therefore return the wrong underlay even when the dump fits into 8192 bytes. For a VPN client the intended lookup is specifically: ```text best IPv4 default route ``` not: ```text first main-table route with a gateway ``` The `rtm_dst_len == 0` and lowest-metric selection in #3052 address this correctly. ## 4. More importantly: failure to install the endpoint exclusion is ignored This is the part I would treat as the critical lifecycle bug. During `Daemon::activate()` current code does: ```cpp for (const QString& i : config.m_excludedAddresses) { addExclusionRoute(IPAddress(i)); } ``` The return value is ignored. So: ```text endpoint exclusion failed ``` does not stop activation. The peer is then configured: ```cpp if (!wgutils()->updatePeer(config)) { ... } ``` and finally the full-tunnel routes are installed. For IPv4 `/0`, Linux intentionally converts: ```text 0.0.0.0/0 ``` into: ```text 0.0.0.0/1 128.0.0.0/1 ``` pointing to `amn0`. Therefore the actual sequence can be: ```text try endpoint /32 exclusion ↓ getGatewayAndIface() overflows ↓ "No default gateway available" ↓ exclusion NOT installed ↓ error ignored ↓ peer configured ↓ 0.0.0.0/1 + 128.0.0.0/1 installed ↓ VPN endpoint now matches a tunnel route ``` At that point the client has knowingly committed a full-tunnel route while its control-plane escape route is absent. That should be a forbidden state. ## 5. `updatePeer()` repeats the same failed operation and also ignores it `WireguardUtilsLinux::updatePeer()` contains another endpoint-exclusion attempt: ```cpp if ((config.m_hopType != InterfaceConfig::MultiHopExit) && (m_rtmonitor != nullptr)) { m_rtmonitor->addExclusionRoute( IPAddress(config.m_serverIpv4AddrIn)); m_rtmonitor->addExclusionRoute( IPAddress(config.m_serverIpv6AddrIn)); } ``` Again the return values are ignored. So there are currently two independent best-effort layers: ```text Daemon::activate() -> exclusion failure ignored WireguardUtilsLinux::updatePeer() -> exclusion failure ignored ``` Neither establishes: ```text VPN endpoint is reachable through underlay ``` before installing full-tunnel routes. I would consolidate ownership of this operation rather than attempting it twice. ## 6. This explains the recursive WireGuard packet growth If the server is, for example: ```text 203.0.113.50 ``` and its physical `/32` exclusion route is missing, one of: ```text 0.0.0.0/1 128.0.0.0/1 ``` wins. The userspace WireGuard process attempts to send its outer UDP packet to: ```text 203.0.113.50 ``` but the kernel routes that packet into: ```text amn0 ``` The packet presented to the tunnel is then treated as traffic that itself needs tunnelling. Conceptually: ```text inner packet ↓ WG encapsulation ↓ outer UDP to VPN server ↓ route says amn0 ↓ outer packet becomes tunnel input ↓ WG encapsulates it again ↓ larger outer packet ↓ amn0 again ↓ ... ``` until packet size limits are reached. That makes: ```text sendmmsg: message too long ``` a very plausible downstream consequence of the missing endpoint escape route, rather than an independent MTU problem. The user's manual workaround: ```sh ip route add <server-ip> \ via <physical-gateway> \ dev <physical-interface> ``` breaking the loop is a strong runtime confirmation of the same path. ## 7. The error path also leaks a netlink socket `getGatewayAndIface()` creates: ```cpp sock = socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE) ``` but `close(sock)` only occurs on the normal path near the end of the function. For example the overflow path reaches: ```cpp perror("Error in received packet"); return {}; ``` without closing `sock`. The same happens for several other early returns. Therefore every failed gateway lookup can leak one netlink FD. The reporter sees approximately ten gateway lookup failures per connection attempt. So repeated reconnects can additionally produce: ```text gateway lookup failure ↓ netlink FD leaked ↓ retry ↓ another FD leaked ``` This should probably use an RAII/scoped FD rather than manually trying to cover every return path. ## 8. #2862 explains the apparent 4.8.x -> 5.0.x regression PR #2862 changed this code from: ```text 100-byte buffer + incorrect multipart handling ``` to: ```text 8192-byte buffer + proper response-header multipart handling ``` and changed parsing from approximately the first receive block to the accumulated dump. So older releases could accidentally succeed because they effectively parsed only an early route message. Once multipart reception was corrected, large route tables started exposing the fixed 8192-byte ceiling. So I would not treat: ```text 4.8.21 works ``` as evidence that the previous implementation was correct. It happened to avoid this failure mode. ## 9. #3052 fixes the main parser bug, but activation should also enforce the endpoint-route invariant I think #3052 is the right direction for the route parser: ```text AF_INET-only dump parse multipart responses incrementally require actual default route respect route metric ``` But even with the parser fixed, I would separately harden the activation transaction. For a single-hop/full-tunnel connection: ```text endpoint exclusion ``` is not optional configuration. It is a prerequisite for making tunnel routes authoritative. The ordering should be approximately: ```text resolve endpoint ↓ resolve usable underlay route ↓ install endpoint exclusion ↓ verify endpoint best route is NOT amn0 ↓ configure peer ↓ install /1 tunnel routes ↓ verify endpoint still resolves to underlay ↓ continue connection ``` If the exclusion cannot be established: ```text activation must fail ``` rather than: ```text warning + continue into a routing loop ``` ## 10. Distinguish parser failure from "no default route yet" There is also an important distinction with cases such as boot-time autoconnect. These are different states: ```text A. RTM_GETROUTE failed / malformed / truncated B. route dump succeeded, but no IPv4 default route exists yet ``` Both currently collapse into: ```text gateway.isEmpty() ``` and then: ```text No default gateway available ``` But the recovery policy is different. For A: ```text internal routing discovery failure ``` For B: ```text underlay not ready yet -> bounded retry / wait for route event ``` It would be useful for `getGatewayAndIface()` to return a typed result instead of encoding all failure classes as an empty `QPair`. ## 11. IPv6 endpoint exclusion is another adjacent problem `updatePeer()` attempts both: ```text serverIpv4AddrIn serverIpv6AddrIn ``` but Linux `rtmSendRoute()` obtains the gateway through the IPv4-only: ```text getGatewayAndIface() ``` and always creates the gateway attribute from: ```cpp struct in_addr ip4; inet_pton(AF_INET, gateway.toUtf8(), &ip4); ``` So the current exclusion construction is not genuinely family-aware. That should probably be separated from #3090, but a future route lookup API should return something like: ```text family gateway interface metric ``` rather than a QString IPv4 gateway plus `QNetworkInterface`. ## Suggested invariants The most important one is: ```text FULL_TUNNEL_ROUTE_COMMITTED => VPN_ENDPOINT_ROUTE_CONFIRMED_ON_UNDERLAY ``` And for route discovery: ```text successful default-gateway lookup => complete netlink transaction AND actual default route AND correct address family AND selected route policy is deterministic ``` ## Regression tests 1. More than 8 KiB of route dump data: ```text Docker / libvirt / many interfaces ``` Expected: ```text default gateway still found ``` 2. More than 16 KiB / much larger dumps. The fix should not merely move the hard ceiling. 3. Large IPv6 table + small IPv4 table: ```text IPv4 default lookup ``` must not process unrelated IPv6 routes. 4. Multiple IPv4 default routes with different metrics: ```text expected = correct best default ``` 5. Specific gateway route appears before the default route. Expected: ```text must not be selected as default ``` 6. Force route-discovery failure while connecting full tunnel. Expected: ```text activation fails before /1 routes are committed ``` Forbidden: ```text warning + continue ``` 7. Default route appears shortly after boot. Expected: ```text bounded readiness/retry ``` rather than either an infinite retry or committing a broken tunnel. 8. Repeated failed gateway lookups. Expected: ```text no netlink FD growth ``` 9. Full tunnel after successful setup: ```text ip route get <VPN endpoint> ``` must return the physical underlay interface. At the same time: ```text ip route get <ordinary public IP> ``` must return `amn0`. 10. Deliberately remove the endpoint exclusion after startup. The connection-health logic should detect loss of the control-plane route rather than silently entering recursive encapsulation. So I would classify #3090 as **confirmed and still present in current dev**. The fixed 8 KiB netlink buffer is the immediate regression trigger on the reported host, but the more important lifecycle problem is broader: **failure to establish the VPN endpoint's underlay route is treated as best-effort, and full-tunnel routes are committed anyway.** That turns a recoverable route-discovery failure into a self-capturing VPN endpoint and complete loss of connectivity.

bsakaguchi commented

2026-10-13 20:42:22 +00:00

Author

Owner

@Plasmoid77 commented on GitHub (Sep 23, 2026):

Another data point, from 4.8.19 (Arch Linux, amneziavpn-bin 4.8.19.0, AmneziaWG). The same log sequence appears there, so this is not only a 5.x regression — it depends on the size of the route table, and an ordinary desktop crosses the 8 KiB limit:

Uplink state	Full `RTM_GETROUTE` dump (AF_UNSPEC, all tables)	Result
Wi-Fi only	7500 B in datagrams `[3768, 3712, 20]`	fits, gateway found, VPN works
Wi-Fi + wired dual-stack LAN	9960 B in datagrams `[3676, 6264, 20]`	over 8 KiB → `No default gateway available, skipping exclusion route` → `sendmmsg: message too long`

ip -4 route show table all | wc -l = 24 and ip -6 route show table all | wc -l = 73 in the failing state — a few Docker bridges/veths and libvirt, plus a LAN that advertises three IPv6 prefixes (native, ULA and an overlay /64) with DHCPv6 /128s and their local entries. Measured with a small Python replica of the request and the fixed-buffer read loop.

One consequence worth handling together with #3052: the app still reports Connected while the endpoint exclusion route is missing, so the user sees a working VPN that passes no traffic. Moving the uplink from wired back to Wi-Fi "fixes" it only because the dump fits again.

Happy to test a build with #3052 on this machine.

@Plasmoid77 commented on GitHub (Sep 23, 2026): Another data point, from **4.8.19** (Arch Linux, `amneziavpn-bin` 4.8.19.0, AmneziaWG). The same log sequence appears there, so this is not only a 5.x regression — it depends on the size of the route table, and an ordinary desktop crosses the 8 KiB limit: | Uplink state | Full `RTM_GETROUTE` dump (AF_UNSPEC, all tables) | Result | | --- | --- | --- | | Wi-Fi only | 7500 B in datagrams `[3768, 3712, 20]` | fits, gateway found, VPN works | | Wi-Fi + wired dual-stack LAN | 9960 B in datagrams `[3676, 6264, 20]` | over 8 KiB → `No default gateway available, skipping exclusion route` → `sendmmsg: message too long` | `ip -4 route show table all | wc -l` = 24 and `ip -6 route show table all | wc -l` = 73 in the failing state — a few Docker bridges/veths and libvirt, plus a LAN that advertises three IPv6 prefixes (native, ULA and an overlay /64) with DHCPv6 `/128`s and their `local` entries. Measured with a small Python replica of the request and the fixed-buffer read loop. One consequence worth handling together with #3052: the app still reports **Connected** while the endpoint exclusion route is missing, so the user sees a working VPN that passes no traffic. Moving the uplink from wired back to Wi-Fi "fixes" it only because the dump fits again. Happy to test a build with #3052 on this machine.

bsakaguchi commented

2026-10-13 20:42:22 +00:00

Author

Owner

@vami7ir commented on GitHub (Sep 23, 2026):

Still reproduces on 5.0.3 (Ubuntu, kernel 6.8, ~34 Docker containers). Route dump ≈ 26.6 KB, of which IPv6 ≈ 25.4 KB comes
from veth* interfaces; without them ≈ 1.1 KB.

A workaround that doesn't need a per-server static route: disable IPv6 on the Docker veth interfaces, so the dump fits
into 8 KiB again and the client installs the exclusion route itself (any server, any network):
for i in $(ls /sys/class/net | grep ^veth); do sudo sysctl -qw net.ipv6.conf.$i.disable_ipv6=1; done
To cover new containers, a udev rule /etc/udev/rules.d/90-veth-noipv6.rules:
ACTION=="add", SUBSYSTEM=="net", KERNEL=="veth*", RUN+="/usr/sbin/sysctl -qw net.ipv6.conf.%k.disable_ipv6=1"
Only helps when the extra IPv6 routes come from Docker, not from the LAN itself. Proper fix is still #3170 / #3052.

@vami7ir commented on GitHub (Sep 23, 2026): Still reproduces on 5.0.3 (Ubuntu, kernel 6.8, ~34 Docker containers). Route dump ≈ 26.6 KB, of which IPv6 ≈ 25.4 KB comes from veth* interfaces; without them ≈ 1.1 KB. A workaround that doesn't need a per-server static route: disable IPv6 on the Docker veth interfaces, so the dump fits into 8 KiB again and the client installs the exclusion route itself (any server, any network): for i in $(ls /sys/class/net | grep ^veth); do sudo sysctl -qw net.ipv6.conf.$i.disable_ipv6=1; done To cover new containers, a udev rule /etc/udev/rules.d/90-veth-noipv6.rules: ACTION=="add", SUBSYSTEM=="net", KERNEL=="veth*", RUN+="/usr/sbin/sysctl -qw net.ipv6.conf.%k.disable_ipv6=1" Only helps when the extra IPv6 routes come from Docker, not from the LAN itself. Proper fix is still #3170 / #3052.

bsakaguchi commented

2026-10-13 20:42:22 +00:00

Author

Owner

@Plasmoid77 commented on GitHub (Sep 23, 2026):

Thanks — you're right. I checked client/core/networkUtilities.cpp at tag 4.8.19.0: BUFFER_SIZE = 100, a single recv() (the loop tests NLM_F_MULTI on the request header, so it stops after the first datagram), and then only the first RT_TABLE_MAIN route carrying both RTA_GATEWAY and RTA_OIF counts. So my 7500 / 9960-byte numbers model the 5.x read loop, not 4.8.19 — please take them as a 5.x data point only.

The line you asked for, taken while the tunnel is up (wired LAN and Wi-Fi both connected):

$ ip -4 route show table all | head -1
0.0.0.0/1 dev amn0 metric 1

A replica of the 4.8.19 logic (one 100-byte recv(), first main-table route with gateway and oif) accordingly finds no gateway in that state. I haven't isolated the route order at the moment of a reconnect, after the /1 routes are removed, so I won't guess at that part.

@Plasmoid77 commented on GitHub (Sep 23, 2026): Thanks — you're right. I checked `client/core/networkUtilities.cpp` at tag `4.8.19.0`: `BUFFER_SIZE = 100`, a single `recv()` (the loop tests `NLM_F_MULTI` on the request header, so it stops after the first datagram), and then only the first `RT_TABLE_MAIN` route carrying both `RTA_GATEWAY` and `RTA_OIF` counts. So my 7500 / 9960-byte numbers model the 5.x read loop, not 4.8.19 — please take them as a 5.x data point only. The line you asked for, taken while the tunnel is up (wired LAN and Wi-Fi both connected): ``` $ ip -4 route show table all | head -1 0.0.0.0/1 dev amn0 metric 1 ``` A replica of the 4.8.19 logic (one 100-byte `recv()`, first main-table route with gateway and oif) accordingly finds no gateway in that state. I haven't isolated the route order at the moment of a reconnect, after the `/1` routes are removed, so I won't guess at that part.

bsakaguchi commented

2026-10-13 20:42:23 +00:00

Author

Owner

@McSmog commented on GitHub (Sep 24, 2026):

I can confirm this issue also affects AmneziaVPN 5.0.3.0 on Linux. My machine has multiple Docker networks and routes. The system had a valid default route, but connecting through the GUI left me without internet access.

The service logged Error in received packet: Numerical argument out of domain and No default gateway available, skipping exclusion route. Disabling split tunneling did not help. I verified that the same AmneziaWG 3.1 server and client profile could pass traffic outside the GUI.

I built the 5.0.3.0 background service with PR #3052 applied and used it with the existing GUI installation. Internet access through the GUI now works. This is a confirmed fix on my machine. Please merge and release it — users should not have to rebuild the service to get a working VPN.

@McSmog commented on GitHub (Sep 24, 2026): I can confirm this issue also affects **AmneziaVPN 5.0.3.0 on Linux**. My machine has multiple Docker networks and routes. The system had a valid default route, but connecting through the GUI left me without internet access. The service logged `Error in received packet: Numerical argument out of domain` and `No default gateway available, skipping exclusion route`. Disabling split tunneling did not help. I verified that the same AmneziaWG 3.1 server and client profile could pass traffic outside the GUI. I built the **5.0.3.0 background service with PR #3052 applied** and used it with the existing GUI installation. Internet access through the GUI now works. This is a confirmed fix on my machine. Please merge and release it — users should not have to rebuild the service to get a working VPN.

bsakaguchi commented

2026-10-13 20:42:23 +00:00

Author

Owner

@itelmenko commented on GitHub (Oct 8, 2026):

Same bug on Ubuntu 24.04, client 5.0.3.0 (released 2026-09-18 — started failing for me around that date).

Setup

Protocol: AmneziaWG (amneziawg-go, interface amn0)
Many Docker bridge networks / routes present (br-*, 172.x.x.x)
Same Wi‑Fi / same server works fine on Android and Windows
Without Docker, Linux client works
With Docker running, after Connect (or reconnect) traffic dies

Symptoms

Client shows Connected
amn0 has large TX, RX = 0
ip route get <VPN_SERVER_IP> returns dev amn0 instead of via the Wi‑Fi gateway
No UDP to the VPN endpoint on the uplink (wlo1) until an exclusion route is added manually
After sudo ip route replace <VPN_SERVER_IP>/32 via dev wlo1, outbound UDP appears on wlo1
Stopping Docker (systemctl stop docker) makes the VPN work again; bringing Docker back and reconnecting breaks it again

Conclusion Matches this issue: with a large routing table (Docker), gateway detection / exclusion route install fails, so the VPN endpoint is routed into amn0. Manual exclusion route or stopping Docker is a workaround. Happy to provide more logs if useful.

@itelmenko commented on GitHub (Oct 8, 2026): Same bug on Ubuntu 24.04, client 5.0.3.0 (released 2026-09-18 — started failing for me around that date). Setup - Protocol: AmneziaWG (amneziawg-go, interface amn0) - Many Docker bridge networks / routes present (br-*, 172.x.x.x) - Same Wi‑Fi / same server works fine on Android and Windows - Without Docker, Linux client works - With Docker running, after Connect (or reconnect) traffic dies Symptoms - Client shows Connected - amn0 has large TX, RX = 0 - ip route get <VPN_SERVER_IP> returns dev amn0 instead of via the Wi‑Fi gateway - No UDP to the VPN endpoint on the uplink (wlo1) until an exclusion route is added manually - After sudo ip route replace <VPN_SERVER_IP>/32 via <gateway> dev wlo1, outbound UDP appears on wlo1 - Stopping Docker (systemctl stop docker) makes the VPN work again; bringing Docker back and reconnecting breaks it again Conclusion Matches this issue: with a large routing table (Docker), gateway detection / exclusion route install fails, so the VPN endpoint is routed into amn0. Manual exclusion route or stopping Docker is a workaround. Happy to provide more logs if useful.

bsakaguchi commented

2026-10-13 20:42:23 +00:00

Author

Owner

@pers1307 commented on GitHub (Oct 9, 2026):

@pokamest @Nethius @vkamn help us, pls.

@pers1307 commented on GitHub (Oct 9, 2026): @pokamest @Nethius @vkamn help us, pls.

bsakaguchi referenced this issue

2026-10-13 21:21:09 +00:00

[PR #1431] feature: added lua codec parameter to awg #2348

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

amnezia-vpn/amnezia-client#1431

No description provided.

Rows
Columns

[GH-ISSUE #3090] Linux 5.0.1.5: exclusion routes never installed — netlink route dump overflows the fixed 8 KiB buffer #1431

1. The fixed 8 KiB netlink dump buffer is still in current dev

2. The request unnecessarily asks for both IPv4 and IPv6 routes

3. The current parser does not actually require a default route

4. More importantly: failure to install the endpoint exclusion is ignored

5. updatePeer() repeats the same failed operation and also ignores it

6. This explains the recursive WireGuard packet growth

7. The error path also leaks a netlink socket

8. #2862 explains the apparent 4.8.x -> 5.0.x regression

9. #3052 fixes the main parser bug, but activation should also enforce the endpoint-route invariant

10. Distinguish parser failure from "no default route yet"

11. IPv6 endpoint exclusion is another adjacent problem

Suggested invariants

Regression tests

5. `updatePeer()` repeats the same failed operation and also ignores it